DOCUMENT RESUME 

ED 355 373 CE 063 199 



AUTHOR 
TITLE 

INSTITUTION 
SPONC AGENCY 



PUB DATE 
CONTRACT 
NOTE 

PUB TYPE 



Sie£er» Nancy; Latkiewicz, John 

Third Project Evaluation Report — Summative for Salt 

Lake Community College* 

Salt Lake Community Coll., UT. 

Office of Vocational and Adult Education (ED), 

Washington, DC* National Workplace Literacy 

Program. 

15 Aug 92 

V198A00091 

102p. 

Reports - Evaluative/Feasibility (1A2) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MF01/PC05 Plus Postage. 

Adult Basic Education; '''Adult Literacy; Evaluation 
Methods ; '^Literacy Education; '^Manufacturing 
Industry; ''^On the Job Training; Outcomes of 
Education; Post secondary Education; Program 
Development; ''Trogram Effectiveness; Program 
Evaluation; Program Implementation 
'^Statistical Process Control ; ''Workplace Literacy 



ABSTRACT 

Through a federal grant, Salt Lake Community College 
(Utah), in conjunction with two area industries, implemented a 
workplace literacy project to serve 225 employees. The training 
included work-related curriculum and instruction in reading, English 
as a Second Language, oral communication skills, written 
communication skills, mathematics, and Pre~Statistical Process 
Control (SPC) . A total of 360 employees participated in AO training 
sessions during the 18 months of the grant's operation. The 
curriculum was developed with the help of advisory committees from 
the two industries (a semiconductor manufacturer and a sheet metal 
fabrication plant making computer casings). Subject matter was 
decided through literacy audits. Checklists, supervisors' rating 
sheets, students' and instructor*' evaluations, personal interviews, 
and anecdotal summaries were used to assess the impact of the 
instruction upon the workers and the companies. This evaluation 
provided "hard" data indicating that the participating employees 
improved measurably in the areas targeted by the assessment. The 
evaluation plan also derived "soft" data in which the workers 
indicated other significant gains, such as increases in reading the 
daily newspaper, increased involvement in their children's homework 
and school activities, and improved self-confidence and self-esteem. 
(Extensive appendixes contain the following: sample reports of class 
results, a conference brochure, and the project director's report, 
including an overview, performance report, project objectives, 
recommendations, a list of dissemination activities, and project 
documents.) (KC) 
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Executive SummBrv 
Purpose 

Through funding from the Department of Education, Salt Lake Community 
College, in conjunction with National Semiconductor Corp, (computer chip 
fabricator) and Natter Manufacturing Inc. (sheet metal fabricator), implemented 
a workplace literacy project to serve 225 employees. The training included work- 
related curriculum and instruction in reading, English as a second language 
(ESL), oral communication skills, written communication skills, math, and Pre- 
Statistical Process Control (SRC). The project director and secretary were 
assisted by seven part-time instructors and a counselor. A total of 360 
employees participated in 40 training sessions within the 18 months of the 
grant's operation. 

National Semiconductor Corp. 

Working with the training coordinator, the project director established an 
advisory committee? to guide the selection, assessment, and progress of the 
curriculum, the participants, and the classes. The advisory committee met every 
two weeks throughout the length of the project. Three hundred twenty-five 
employees (out of a workforce of 1150) attended 33 sessions in Pre-SPC, ESL, 
Refresher Math, and Technical Reading/Writing. The majority of the employees 
enrolled in the training exceeded grant objectives in spite of a downturn in the 
economy which resulted in numerous plant shut-downs during the grant's operating 
period. Workers attended classes twice a week for two hours on company time; 
that is, National Semiconductor encouraged participation by giving each employee 
total release time to attend classes. Each instructional session included whole 
group activity as well as individualized self-paced learning. Instructors 
developed a working curriculum based on the results of the literacy audit and 
encouraged class participants to provide the specific examples and incidents to 
make the curriculum relevant to their jobs. A curriculum writer assisted the 
instructors in the final editing and in establishing a common format for the 
curriculum. A variety of instruments was used to evaluate the participants, the 
classes, and the program as a whole. Project participants designed and utilized 
customized assessments as opposed to using standardized tests or measurements. 
Evaluation was multi-level; participants at all levels— workers, 
supervi sors/managers , instructors , evaluators— evaluated themsel ves , the 
instructors, the course, and the program. Each participating worker had a file 
which included: SLCC admission's form; pre/post tests; individualized education 
plan; and representative assignments completed by the employee during the 
program. 

Natter Manufacturing, Inc. 

Natter Manufacturing is a sheet metal fabrication plant whose primary 
product is the production of computer casings. At the project's inception. 
Natter employed 250 workers but due to poor economic conditions locally and 
nationally, Natter's workforce fell to 125 by the end of the program. (In recent 
months Natter has experienced a mild resurgence and has brought back many of its 
laid-off employees.) Using the results of the literacy audit, the project 
director and the advisory committee, which met on an as-need basis, recommended 
classes in refresher and advanced math, communication, and ESL. The extensive 
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work-related ESL courses ranged from basic to advanced and included conversation 
and reading/writing. Thirty-five employees participated in the train ina J^k nJ 
primarily ESL classes and often taki'ng 'more than one course*''worker ^^J^^^e^^ 
^Ih L./''''^ ? 'tl^^^^'' two hours with workers donating one hour of Iheir Sime 
and Natter contributing one hour of company time. Instructors with ass stance 

JorVhp''M*"^'"*' '^^^ a grant-supported curriculum writer 5e;e^iped maJeHS" 
for the math, commumcation, and various levels of ESL. Natter like NaHonil 
Semiconductor, utilized a variety of customized instruments to eSa 22?e 

wh^?r"^rhTM • V '^'^'"^ °f instruction and the p?og?am 

Checklists, supervisors' rating sheets, students' and instructors' 

his/her area ^ workplace instruction upon the individual worker and 



his/her area. 

Conclusion 



The project brought workers, supervisors, and manaaers toaether with 
educational specialists to plan and implement a workplace educatioHr^^^^^ 
intensive and extensive evaluation plan'provided "hard" data S "ndiS'tha! 
a seSnt'.'P'Vh"^ '""^^^y^" measurably' improved in the areas targeted by She 
?nH^^!? evaluation plan also elicited "soft" data in which the workerl 

indicated other significant gains not often measured by standard procedures Fo? 
example, the majority of participants cited an increase in read ina the dailv 

KSnJr ' P^^Q^^'"' ^'"^^^^^^d involvement in thei^^hildJen's 

nlwl activities, and improved self-confidence and self-esJeSm 

wS^kpl^ace litera^^tT^nr^^^^^^^^^^ ^^"'"'^■^ >te^^O""ection between impJove^ 
woFKp I ace literacy and increased involvement in general literacy event*; wpII 
as participation in family literacy activities. 'i^eracy events as well 
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Introduction 



On January 5-8, 1992, Dr. Nancy Siefer and Dr. John Latkiewicz conducted 
the last in a sequence of three project evaluations of the Workplace Literacy 
Project funded by the U.S. Dept. of Education. The evaluation consisted of a 
review of project documentation and materials and meetings with the principals 
which included: 

Julie Van Moorhem, Project Director, SLCC 
John Anjewierden, Division Chair 

Joan Rond, Training Coordinator, National Semiconductor 
Joy Spindler, Personnel Manager, Natter Manufacturing 

Project Overview 

Salt Lake Community College, in conjunction with National Semi-conductor 
Corp* (semi-conductor manufacturer) and Natter Manufacturing (sheet metal 
fabrication - computer casings), conducted a workplace literacy project designed 
to serve 225 employees of the companies involved. The training conducted 
included workplace literacy training related to reading, English as a second 
language, oral communication skills, written communication skills including 
technical writing, math and pre-Statistical Process Control. The project began 
on April 1, 1990 and, as the result of a no-cost project extension, ended on 
January 31, 1992. The budget for the project was $392,733. 

Evaluation Elements 

The elements of the evaluation included assessment of: 

1. the procedures for documentation of activities related to the formal 
project objectives stated in the grant, 

2. the implementation strategies being used and their effectiveness, 

3. the project as it relates to time lines stated in the grant application, 

4. the Collegers and the businesses' commitment to and level of involvement 
with the project, and 

5. procedures and techniques for assessment and testing of 
participants. 

Activities and Observations Related to the Evaluation Elements 

1. the procedures for documentation of activities related to the formal 
project objectives stated in the grant 

Files were maintained on each individual trainee. Included in each file 

were: 

a standard College admissions form, 
pre- and post- tests. 
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an Individualized Education Plan (lEP), and 

representative and significant assignments completed by 
trainees as part of the program. 

Sample files were provided as part of the first two reports. Rolls were also 
maintained which document attendance. The project director maintains separate 
files related to each objective listed in the project's evaluation plan (pp. 27- 
29 of the original grant application). 

Documentation of efforts was a major focus of activities. The project ran 
into problems typical of demonstration projects (i.e., objectives formulated 
before implementation of a class change as the class is conducted, instructors 
focus on teaching first and documentation second; documentation sometimes a 
second priority). In an attempt to ensure that the appropriate data was 
collected, staff were in-serviced on evaluation by the internal evaluator and a 
format for presenting the results of each class developed and used to report 
results. (A sample report is included as Exhibit A.) The format allowed the 
results of individual classes to be articulated in terms of the product 
objectives and to make comparisons between classes. T;iese reports, and the 
supporting documentation, in fact provided the basis for the conclusions reached 
related to achievement of the product objectives. 

2. the implementation strategies being used and their effectiveness 

3. the project as It relates to time lines stated In the grant application 

The evaluation report as it relates to the two elements listed above is 
provided on a project objective by project objective basis. Since the evaluation 
elements are interrelated, they are jointly covered in the discussion of the 
process objectives. 



Process Objective 1: Conduct literacy audit 

Although verification and modification of the literacy audit results was 
an on-going process as curriculum is developed, the objective was satisfied as 
described in the previous report. On-going efforts include meeting with the 
company curriculum committees and individual supervisor to refine curriculum. 



Process Objective 2: Present audit results to management. 

The objective has been met. Results were described in the previous report. 



Process Objective 3: Perform tasfc analysis for selected Jobs. 

The objective has already been accomplished and the accompl ishments 
described in the previous report. On-going refinements were made. Efforts 
include meeting with the company curriculum committees and individual supervisors 
to refine curriculum. 
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Process Objectives 4 & 5: Design Curriculum I Develop Curriculum. 

Curriculum was developed in the areas of: 

- at National Semiconductor - 

Pre-SPC training, 
Refresher/Stepper Math, 
ESL Level 1, 2 and 3, 
Team Building, and 
Technical Reading/Writing; 

- at Natter Manufacturing - 

Advanced Math, 

Vocational English for Special Purposes, 
Refresher Math, 
Workplace Communications, and 
Technical Reading/Writing. 

Curriculum has been designed and executed for all courses. The project 
hoped to have curriculum developed which: 

followed a common format, 

was complete in terms of that format, 

was systematic, and 

with minor modification, could be taken off the shelf and used by 
instructors of future sections of the course or by other workplace 
literacy projects. 

Although all instructors developed curriculum and although, for the most part, 
the developers met the above criteria, the curriculum cannot, as a whole, be said 
to meet the above criteria completely. The chart which appears on the page 7 
provides an analysis of the curriculum. The columns on that chart deal with: 



Objectives 

WPL oriented 
quality overall 

Pre- and Post Tests 

WPL oriented 

pre-test developed, quality overall 
post-test developed, quality overall 

Lesson Plans 

WPL oriented 
quality overall 
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Class Activities/Materials 

WPL oriented 
quality overall 

Course Evaluations 

WPL oriented 
quality overall 

As the chart indicates, the attempt to systematize the curriculum and ensure a 
WPL orientation succeeded for the most part for but not entirely. Suggestions 
for improvement of the curriculum design process are made in the Recommendations 
section. 



Process Objective 6: Implement the program. 

The project staff included: 

SLCC 

Julie Van Moorhem Project Director 

Mary Mellott ESL Instructor/Assist. Dir. 

Nancy Fill at ESL Instructor 

Julia Balcom Math Instructor/Coordinator 

Beth Malloy Math Instructor 

Don Allen Math Instructor 

Mark Merkley Communication Instructor 

Audrey Nelson Reading/Writing Instructor 

Monique de la Mare Counselor 

Shanna Dodge Secretary 

Arnette Starks Secretary (replaced Shanna Dodge as of 9/3/91) 

John Anjewierden Division Chair (supervisor immediately over 

the program) 

The main industry contacts for the program were as follows: 

National Semi-Conductor 
Joan Rond Training Coordinator 

Natter Manufacturing 
Joy Spindler Personnel Manager 
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The classes implemented include: 



National Semiconductor 



Class 



Number 
Trained 



Sessions 
Conducted 



Pre-SPC Training 
ESL 

Refresher Math 
Technical Reading/ 



128 
28 

154 
15 



17 
3 

12 
1 



Writing 
Team Building 



14 



1 



Natter Kanufacturing 



ESL I 

ESL II 

ESL Basic 

ESL Conversation 

ESL Reading/Writing 

Workplace Communication 

Technical Reading/Writing 

Refresher Math 

Advanced Math 



10 
8 
4 
7 
6 

19 
7 

25 

10 



2 
2 
1 
1 
1 
2 
1 
3 
1 



As noted, documentation related to the training included individual trainee 
files (admissions paperwork, pre- and post- tests, and major assignments), 
attendance rolls and trainee evaluations of the trainer and the class. 
Evaluations conducted produced a favorable rating of all classes. The results 
of the evaluations correspond to the reports from supervisors and other company 
personnel all of which are favorable. 

In terms of the numbers and focus of the training, the objective has been 
met. All content areas have been addressed. The numbers trained have exceeded 
those stated as part of the product objectives. As will be outlined as part of 
the product objectives, the implementation has been successful. 

Although the project has succeeded, success according to the time line has 
been problematic. Because of lay-offs and other unforeseen events, the project 
ran behind schedule in terms of the classes offered. With the no-cost extension 
allowed by U.S.D.E., the project has, however, met the grant's objectives. This 
particular process objective can be considered to have been met for the most 
part. 

Two sub-activities were not accomplished and need to be noted. The first 
relates to the "tutoring" program which was proposed as part of the project. The 
intent was to utilize student volunteers from the University of Utah's Pennion 
Community Services Center. The one factor that was not taken into account was 
the location of the companies involved relative to the University. They are for 
all practical purposes at different ends of a 30 mile by 30 mile valley. Public 
transportation is also quite limited. Traditionally, the area represented by the 
com^jdnies is underrepresented at the University. The volunteer pool to serve the 
companies was therefore non-existent. 
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The other problematic area was the "conference" to be hosted by the 
project. This particular effort is discussed Under Process Objective 8: 
Disseminate results. 

Process Objective 7: Monitor/ evaluate the program. 

The first and second formative reports anri this last summative evaluation 
themselves serve as documentation of the accomplishments related to this 
objective. Resumes for both the external and internal evaluator were included 
as part of the first formative evaluation report. On an on-going basis, the 
internal evaluator has been involved with set-up of the project management 
function including the documentation procedures, review of materials and periodic 
reviews of the project. The objective has been met. 

Process Objective 8: Disseminate results. 

The strategies related to this objective were largely acted upon during the 
last six months of the project. The project director and her staff planned the 
major dissemination effort, a conference to have been held in November of '91. 
All reasonable steps were taken to develop the conference. These include 
establishment of conference format and schedule, arrangements for speakers, 
design and distribution of a quality brochure (included as part of Exhibit B) and 
promotional activities supported by the College's Office ot Workshops, Seminars 
and Conferences including mailed brochures and news articles. Despite these 
efforts, only 16 individuals actually signed up for the conference. Based upon 
the limited registration, the conference was cancelled. 

In addition to the conference, the project has disseminated information 
through participation in the following conferences: 



"Building Industrial Leaders through Literacy" 

Mountain Plains Adult Education Association Conference 
November 1991 - Sun Valley, Idaho 
Mary Mellott and Julia Balcom 

"Gaining Supervisor Support" 

Mountain Plains Adult Education Association Conference 
November 1991 - Sun Valley, Idaho 
Julie Van Moorhem 

"Workforce Literacy: What Are We Doing?" 

June Vocational/Applied Technology Conference 
June 1991 - Logan, Utah 
Jul ie Van Moorhem 

"Workplace Literacy in the 90* s" 

Utah Association for Adult, Community and Continuing Education 

Conference 
Oct-ber 1991 - Park City, Utah 
Jul ie Van Moorhem 
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"Workplace Literacy: Partnerships for Success" 

Midwest Regional P.^ading and Study Skills Conference 
March 1992 - Kansas City, Missouri 
Julie Van Moorhem 

"Workplace Literacy: Partnerships for Success" 

Mountain Plains Adult Education Association Conference 
May 1992 - Salt Lake City, Utah 
Julie Van Moorhem 

In addition. National Semi-Conductor ' s training coordinator, the project director 
and internal evaluator prepared a manuscript for publication entitled Job-Linked 
Literacy Programs: Innovative Strategies at Work (Work in America Institute, 
Spring '92). 

Product Objectives 1 - 8: 

In consultation with the evaluation team, the project Director has prepared 
a detailed report dealing with each of the objectives. The report is complete 
and a restatement of the report here by the evaluation team would not be anything 
more than redundant. Instead, a copy of the report is included as part of the 
evaluation (Exhibit C). The objectives, taken as a whole, were achieved. As the 
report details, some problems existed with regard to achievement of results and 
documentation of results. Given the scope of effort, however, the project 
clearly has met both the intent and the spirit of the objectives if not each 
letter in each objective. 

4. the Conege's and the businesses' conmitment to and level of involvement 
with the project 

As in the first formative evaluation, each of the three partners was asked 
to provide a brief summary of their contributions related to the match outlined 
in the grant and any other contributions that have been made beyond the promised 
match. These summaries have been provided in Exhibit D. 

In more general and global terms, commitment was demonstrated through 

forming active advisory committees at each company, 

making arrangements for personnel at all levels to meet 
with College staff as needed for curriculum development 
or forming curriculum teams, and for providing data, and 

providing of release time and/or overtime in order to 
facilitate employee participation in the program. 

5. assessment and testing of participants 

Assessment was a struggle for the project. The general project objectives 
provide a guideline for the course objectives but do not give clear direction for 
individual courses. Even with the data from the original task analyses, the 
instructors found that they needed to refocus their courses as they went along. 
Much of this was attributable to the newness of the staff to workplace literacy 
training and the inability of supervisors to clearly articulate needs. Although 
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the evaluators observed increasing facility with need analysis over the course 
of the project, the project continued to be a learning process for both 
instructors and supervisors clear until the end of the grant. 

As a result of the above problems, course objectives were somewhat of a 
moving target. The upshot of these problems was that it was difficult to develop 
true pre- tests which encompassed all of the objectives or a post-test which 
corresponded to the pre-test in a meaningful way. To address this issue, the 
evaluators recommended that pre-tests be developed as units within courses as the 
courses became more clearly defined. The evaluators also recommended that the 
assessment procedures (as well as the curriculum) be developed with the ultimate 
program objectives clearly in mind. 

To the credit of the project, the above problems stem in part from a 
commitment of the project and its personnel to develop course specific 
evaluations rather than relying on off-the-shelf, generic skill tests. 



General Observations - Strengths. Areas for Improvement and Recommendations 

As it relates to strengths 

The project director and her staff were committed to conducting a 
true workplace literacy ^jrogram. Where problems occurred in terms 
of time line, it was in part due to the fact that the project was 
trying to do "it" (workplace literacy training) right the first time 
rather than taking short-cuts or skimping on workplace content. 

The experience and background of the project director and project 
staff were readily apparent. The project director possessed a mix 
of expertise in literacy and adult education, curriculum development 
ski 1 Is and management ski 1 Is. Her abi 1 ity to work with the 
companies and maximize the opportunities presented and minimize the 
problems provided maximum return on the federal and local dollars 
involved. 

The project staff were held in high esteem by the company 
supervisors and managers. Student evaluations were likewise 
favorable. 

The participants were committed to the project. This was despite 
economic downturns which plagued both companies. 

The project was institutionalized. The project developed a formal 
relationship with the College's Center for Business and Economic 
Development. CBED will continue to conduct outreach to business and 
industry on behalf of workplace literacy. 

The College has been and continues to be committed to the project. 
The College is attempting to expand the grant effort both through 
additional U.S.D.E. funding and standard contracts for services. 
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As it relates to areas for improvenient . 

A better understanding on the part of College administrators as to the 
parameters of the grant and the constraints of the grant. 

A more systematic approach to needs analysis, to incorporating the results 
into the design of the curriculum and to the tieing of curriculum to that 
needs analysis. 

Development of a guide for and/or system to the development of curriculum 
(preferably based upon a competency-based education model). 

Development of a guide for the documentation of curriculum and of 
instructional activities. 

Development of evaluation procedures related more closely to the 
objectives of the grant. 

In-servicing of administration and staff as to the above items. 
Improveu communication within the companies. 
As it relates to overall recommendations . 

Maintain the focus on workplace literacy. The grant afforded an 
opportunity for in-depth needs assessment and curriculum development. The 
challenge will be to maintain a focus on quality and not short-change the 
companies and their employees (nor allow the companies to short change 
themselves) by dressing up generic literacy approaches and selling them as 
workplace literacy. 

Hire and maintain a curriculum expert. The odds are that the majority of 
the instructional staff will not possess in-depth expertise in curriculum 
design and development nor in evaluation of curricula. 

Maintain an infrastructure (equipment and personnel (director, curriculum 
developer and secretary)) which can support workplace literacy efforts. 

Develop additional and alternative communication techniques for and 
within the companies, for example, newsletters and all-personnel or 
all -management meetings. If the project staff learned one thing, it 
was that company commitment at the line-level supervisor level is an 
on-going but critical struggle. 

Develop formats and protocols that ensure in-depth needs analysis, 
systematic development of curricula, the tieing of curricula and 
evaluation techniques to clearly articulated objectives, and 
systematic collection of useable and useful data. 

Evaluation Summary 

The project was a success. The delays experienced were largely the result 
of the uncontrollable economic circumstances of the business partners and the 
typical overly optimistic start-up plan. With the no-cost extension, the project 
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has successfully addressed both the process and the product objectives with only 
minor short-falls. The benefits of the project to the trainees, the companies 
and the College have, however, out-weighed these minor short falls. Both the 
business partners and the College have been fully committed to the project. The 
challange will be to maintain the commitment in the absence of outside funding 
such as that provided by the federal grant. 
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Exhibit A - Sample - Reports by Instructors on Course 
Exhibit B - Conference Brochure 
Exhibit C - Product Objectives Report 
Exhibit D - Partner Commitments 
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Exhibit A - Sample - Report by Instructors on Courses 



English as a Secon d Language (ESL) Course Evaluation Summaries 

Natter Manufacturing, Inc., 1990-1991 
Instructor: Mary Mellott 

L Course description 

The English as a Second Language classes at Natter Manufacturing 
are a part of the Workplace Literacy Training Project conducted by Salt 
Lake Community College. The ESL student/employees volunteer to attend 
classes. The company excuses the employees from one hour of work for 
each class and the employee donates one hour of their own time for each 
class. The classes meet two days a week, two hours each class. Most of 
the classes were scheduled with twelve week segments and the students 
at Natter were encouraged to continue from one twelve week segment into 
the next ESL segment. 

All ESL students who expressed an initial interest in the program by 
signing the sign-up list, were given an individual assessment designed to 
determine their proficiency levels in reading and writing, and in speaking 
and listening tasks. The assessments v/ere administered by the ESL 
instructors prior to the beginning of each course. All assessments were 
given during work hours at the companies and took approximately 30 
minutes. Based on the results of these assessments, students/employees 
were divided into appropriate level classes. 

A Basic Level class was designed for three interested students 
who were pre-literate in English. This class met only two 
hours a week due to scheduling difficulties. 

ESL I class was designed for the intermediate skill level 
employees. This class met for a total of four hours each week. 

ESL 2 class was designed for the most advanced skill level 
employees. This class met for a total of four hours each week. 

ESL Conversation class was designed to improve conversation 
skills in order to increase social and work related communication. 
This class met for a total of four hours each week. 



II. Course Participants 



ESL students were selected for participation in the grant based on 
their English language proficiency. All students were non-native speakers 
of English, born outside of the United States. Their education levels 
varied from no previous formal education to completion of post-secondary 
programs. An assessment was administered to all interested/prospective 
employees, and selection was based on individual needs. Participation in 
all ESL classes was voluntary. 



Attendance and Retention 



Four ESL classes were taught at Natter. 



ESL Bar !c: This course ran from November 19, 1990, through May 
22, 1991, on Mondays and Wednesdays from 1:30-2:30 p.m. 

4 students enrolled 

3 students completed and received certificates 
75% Retention 

En:ployment was terminated for the orte student not completing this 
course. 



Attendance: Actual Hours of Attendance = 1 02 



Possible Hours of Attendance 



120 



Rate 



85% 
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August 5, 1992 



ESL Level 1 : This class ran February 28, 1991, through May 23, 
1991, on Tuesdays and Thursday from 2:30-4:30 p.m. 

9 students enrolled 

4 students completed and received certificates 
1 student continued but did not receive certification due to lack of 
attendance 

55% Retention 

Of the 4 students who did not complete the course, all of them could not 
participate in the second hour of class for personal reasons. Company 
policy required attendance during both hours. 

Attendance: Actual Hours of Attendance = 245 
Possible Hours of Attendance = 328 

Rate = 75% 



ESL Level 2 : This class ran February 25, 1991, through May 22, 
1991, on Mondays and Wednesdays from 2:30-4:30 p.m. 

4 students enrolled 

4 students completed and received certificates 
100% Retention 

Attendance: Actual Hours of Attendance = 176.5 

Possible Hours of Attendance = 192 

Rate = 92% 
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ESL Conversation : This class ran July 9, 1991, through September 
19, 1991, on Tuesdays and Thursdays from 2:30-4:30 p.m. 

7 students enrolled 

5 students completed and received certificates 
1 student continued but did not receive certification due to lack of 
attendance 

85.71% Retention 

One student did not complete the course due to lay-off. 

Attendance: Actual Hours of Attendance = 184 

Possible Hours of Attendance = 230 

Rate = 80% 



III. Course Content 

The curriculum design was performance- based within the functional 
context of the employees jobs. The results of task analysis and 
interviews with company managers and supervisors were used to prepare 
course outlines. The ESL I and 2 level classes had basically the same 
curriculum outline based on various workplace materials and employee 
situations appropriate for expanding English usage on the job. The 
difficulty level of the content was based on criterion-referenced pre- 
testing scores. The extent of students' mastery of course materials was 
determined by pre- and post-testing scores when appropriate. 

The daily class format included reviews of Natter workplace written 
materials used by the employees, workplace vocabulary enhancement, role- 
play scenarios depicting actual situations employees encounter on the job, 
pronunciation and grammar instructions that were necessary to support 
the employees' continuing English usage, and computerized individual 
reading assignments to increase general reading levels in English. 
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The course content materials: move tags, assembly vocabulary, 
employee performance reviews, shop floor order travelers (routers), and 
level one statistical process control information, were selected as 
necessary job-related documents suitable for adaptation in the ESL 
curriculum. 

The ESL Conversation course was designed to improve 
conversation skills in order to increase social and work related 
communication. By improving verbal, non-verbal, and listening skills 
through role-play and workplace simulations, oral communication skills 
could be enhanced. The course content outline included- polite listening 
skills, non-verbal communication, interviewing .skills, and individual 
student presentations. All units were selected as necessary job-related 
communication tasks. 



IV. Project Objectives Targeted by Course 

ESL Basic : Four students enrolled in the class and three completed 
the first course. A meaning-centered approach based on the use of 
language experience stories, which employed the learner's own words and 
ideas was used with this class. Students were encouraged to take an 
active role in producing materials for class practice. Pretesting based on 
student-produced language experience stories was not feasible as the 
content of the material was unknown prior to the class. Repetition and re- 
cycling of the material was necessary and the time needed for mastery 
varied deperiding on the particular objective and the individual student. 

Samples of student class materials indicate that improvements 
were made in every aspect of language learning. Their primary 
achievements have been to successfully fill out routine personal 
identification information forms and to recognize and respond 
appropriately to company safety signs. Supervisor reports validate the 
gains these students have made; they show an increased confidence in 
using the language on the job and have also increased participation in job- 
related social situations. 
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ESL Level 1 : Nine students enrolled in the class and four 
completed the course. The pre- and post-test averages indicate that the 
class members achie\^ed 80 percent mastery on two of the five unit tests 
that were given during the class. Average class gains of 43 percent and 
47 percent were achieved on two other unit tests. 

Pre/Post Test One - - 5 students took the pre- and post-test. The 
(Using a Move Tag) • class average increased from 94 percent on 

the pre-test to 100 percent on the post-test, 
an average gain of 6 percent. The high scores 
on the pre- and post-testing reflect the 
students prior knowledge of this printed 
workplace form. The move tag is used in 
conjunction with other written workplace 
materials and its mastery is a necessary 
component of the job. Knowledge of the move 
tag relates to Product Objective 1: increase 
workplace literacy levels to prepare for 
advancement opportunities; Product 
Objective 3: increase ability to follow 
detailed sequential directions; Product 
Objective 4: increase recognition/ knowledge 
of workplace vocabulary and abbreviations; 
Product Objective 7: increase document 
literature related to use of manuals, work 
orders, etc.; and Product Objective 8: 
increase workplace writing skills. 



Pre/Post Test Two - - 3 students took the pre- and post-test. The 
(Assembly Vocabulary) class average increased from 17 percent on 

the pre-test to 60 percent on the post-test, 
an average gain of 43 percent. The assembly 
vocabulary was specified by the company and 
contained items quite difficult for this ESL 
level. Although the 43 percent average gain 
was impressive and indicates acquired 
knowledge, the starting scores were very low 
and therefore the 80 percent required for 
mastery was not achieved. This content 
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material relates to Product Objective 1: 
increase workplace literacy levels to prepare 
for advancement opportunities; and Product 
Objective 4: increase recognition/ knowledge 
of workplace vocabulary and abbreviations. 

Pre/Post Test Three - - 3 students took the pre- and post-test. The 
(Employee Performance class average increased from 24 percent on 
Reviews) the pre-test to 71 percent on the post-test, 

an average gain of 47 percent. The vocabulary 
contained in the company's employee 
performance reviews was rather advanced 
for this ESL level. Although the 47 percent 
average gain was impressive and indicatts 
acquired knowledge, the starting scores were 
very low and therefore the 80 percent 
required for mastery was not achieved. This 
content material relates to Product 
Objective 1: increase workplace literacy 
levels to prepare for advancement 
opportunities; Product Objective 4: increase 
recognition/ knowledge of workplace 
vocabulary and abbreviations; and Product 
Objective 7: increase document literature 
related to use of manuals, work orders, etc. 

Pre/Post Test Four - - 4 students took the pre- and post-test. The 
(Router Abbreviations) class average decreased from 88 percent on 

the pre-test to 84 percent on the post-test, 
an average loss of 4 percent. This loss can 
most likely be attributed to differences 
between the pre- and post-test design, as 
well as a lenghty delay between the two 
tests. The scores do indicate the students 
mastery of the material that is an essential 
part of their jobs. The router material 
relates to Product Objective 1: increase 
workplace literacy levels to prepare for 
advancement opportunities; Product 
OLjective 3: increase ability to follow 
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detailed sequential directions; Product 
Objective 4: increase recognition/ knowledge 
of workplace vocabulary and abbreviations; 
and Product Objective 7: increase document 
literature related to use of manuals, work 
orders, etc. 

Pre/Post Test Five - - 4 students took the pre- and pcst-test. The 
(Statistical Process class average increased from 53 percent on 

Control) the pre-test to 65 percent on the post-test, 

an average gain of 12 percent. This material 
was primarily company text that contained 
more advanced technical vocabulary. The 
instructor designed pre- and post-tests were 
not as significant for the students to master 
as the company designed SPG computer 
quizzes. Three of the four completing 
students also passed the company criteria 
(80 percent) for Statistical Process Control, 
Level One. They each received a Certificate 
of Accomplishment from the company. The 
SPC material relates to Product Objective 1: 
increase workplace literacy levels to prepare 
for advancement opportunities; Product 
Objective 3: increase ability to follow 
detailed sequential directions; Product 
Objective 4: increase recognition/ knowledge 
of workplace vocabulary and abbreviations; 
and Product Objective 7: increase document 
literature related to use of manuals, work 
orders, etc. 



Product Objective 5: increase communications skills appropriate to the 
workplace, was met by daily class exercises and practices involving 
students in workplace simulations, role-play, and social conversations 
appropriate to the working environment. Improvements were noted by 
supervisors and other company personnel by reporting anecdotal data. 
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Product Objective 6: increase participation in team meetings, was not 
designed into the ESL class curriculum. 

Product Objective 8: increase workplace writing skills, was met by 
regular exercises in the kind of writing needed to complete company 
forms used by the students on their jobs. This writing was primarily 
limited to transfer of information tasks. Improvements were noticed by 
supervisors and other company personnel by reporting anecdotal data. 

Product Objective 9: increase quantitative literacy skills, was not 
designed into the ESL class curriculum. 



ESL Level 2 : Four students attended and completed this class. The 
pre- and post-test averages indicate that all four members achieved at 
least 80 percent mastery on the five unit tests that were given during the 
class. 

Pre/Post Test One - - 3 students scored 100 percent on pre; 2 
(Using a Move Tag) students scored 100 percent on post (only 1 

student took both the pre- and the post-test). 
The high scores on the pre- and post-testing 
reflect the students' prior knowledge of this 
printed workplace form. The move tag is 
used in conjunction with other written 
workplace materials and its mastery is a 
necessary component of the job. Knowledge 
of the move tag relates to Product 
Objectivel: increase workplace literacy 
levels to prepare for advancement opportu- 
nities; Product Objective 3: increase ability 
to follow detailed sequential directions; 
Product Objective 4: increase recognition/ 
knowledge of workplace vocabulary and 
abbreviations; Product Objective 7: increase 
document literacy related to use of manuals, 
work orders, etc.; and Product Objective 8: 
increase workplace writing skills. 
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Pre/Post Test Two - - All 4 students took the pre- and post- 
(Assembly Vocabulary) test. The class average Increased from 

38 percent on the pre-test to 83 percent on 
the post-test, an average gain of 45 percent. 
The 45 percent average gain was impressive 
and indicates the students increased their 
knowledge of this content material and the 
80 percent required for mastery was 
achieved. This content material relates to 
Product Objective 1: increase workplace 
literacy levels to prepare for advancement 
opportunities; and Product Objective 4: 
increase recognition/ knowledge of workplace 
vocabulary and abbreviations. 



Pre/Post Test Three - - All 4 students took the pre- and post- 
(Employee Performance test. The class average increased from 

Reviews) 68 percent on the pre-test to 89 percent on 

the post-test, an average gain of 21 percent. 
The 21 percent gain indicates the students 
increased their knowledge of this content 
material and achieved the 80 percent score 
required for mastery. The content material 
in this unit relates to Product Objective 1: 
increase workplace literacy levels to prepare 
for advancement opportunities; Product 
Objective 4: increase recognition/ knowledge 
of workplace vocabulary and abbreviations; 
and Product Objective 7: increase document 
literature related to use of manuals, work 
orders, etc. 



Pre/Post Test Four - - All 4 students took the pre- and post-test. 

(Router Abbreviations) The class average increased from 80 percent 

on the pre-test to 97 percent on the post- 
test, an average gain of 17 percent. The 
scores reflect the students mastery of 
content material that is an essential part of 
their jobs. The router material relates to 
Product Objective 1: increase workplace 
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literacy levels to prepare for advancement 
opportunities; Product Objective 3: increase 
ability to follow detailed sequential 
directions; Product Objective 4: increase 
recognition/ knowledge of workplace 
vocabulary and abbreviations; and Product 
Objective 7: increase document literature 
related to use of manuals, work orders, etc. 

Pre/Post Test Five - - All 4 students took the pre- and post- 
(Statistical Process test. The class average increased from 75 

Control) percent on the pre-test to 83 percent on the 

post-test, an average gain of 17 percent. All 
four students also passed the company 
criteria (80 percent) for SPG, Level One. 
They each received a Certificate of 
Accomplishment from the company. The SPC 
material relates to Product Objective 1: 
increase workplace literacy levels to prepare 
for advancement opportunities; Product 
Objective 3: increase ability to follow 
detailed sequential directions; Product 
Objective 4: increase recognition/ knowledge 
of workplace vocabulary and abbreviations; 
and Product Objective 7: increase document 
literature related to use of manuals, work 
orders, etc. 

Product Objective 5: increase communications skills appropriate to the 
workplace, was met by daily class exercises and practices involving 
students in workplace simulations, role-play, and social conversations 
appropriate to the working environment. Improvements were noted by 
supervisors and other company personnel by reporting anecdotal data. 

Product Objective 6: increase participation in team meetings, was 
not designed into the ESL class curriculum. 

Product Objective 8: increase workplace writing skills, was met by 
regular exercises in the kind of writing needed to complete company 
forms used by the students on their jobs. This writing was primarily 
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limited to transfer of information tasl<s. Improvements were noted by 
supervisors and other company personnel by reporting anecdotal data. 

Product ObjectivG 9: increase quantitative literacy skills, was not 
designed into the ESL class curriculum. 

ESL Conversation 

Pre/Post Test One: - Four students took the pre- and post-test. 

(Polite Listening) The class average increased from 78 on the 

pre-test to 81 percent on the post-test, an 
average gain of 3 percent. This score 
indicates the students mastery of the content 
material. Knowledge of polite listening 
behavior relates to Product Objective 5: to 
increase communication skills appropriate to 
the workplace. 

Five students took the pre- and post-test. 
The class averace decreased from 84 percent 
on the pre-test to 75 percent on the post- 
test, and average loss of 9 percent. This 
score is not representative of the total class 
knowledge of non-verbal communication. Two 
students had significant decreases in their 
post-test scores, most likely due to a mis- 
understanding of the post-testing procedure. 
These low scores account for the decreased 
class average. Non-verbal communication 
relates to product objective 5: to increase 
communication skills appropriate to the work- 
place. 

V. Relation of Course to the Product Objectives and Task 

Angiygig 

The learning outcomes In the ESL classes were intended to parallel 
the Product Objectives specified in the grant. 



Pre/Post Test Two ~ 

(Non-Verbal 

Communication) 
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For ESL I and 2 classes, Statistical Process Control materials 
were adapted into the curriculum with non-native English speakers in 
mind whenever it was feasible. This content material is designed to 
relate to Product Objective 2: increase workplace literacy levels to 
prepare for technology upgrades. 

Pre-tests were administered to determine the proficiency levels of 
employees' document literacy. Curriculum materials were designed using 
manuals, work orders, catalogs, and other work-related documents and 
were intended to relate to Product Objectivel: increase workplace 
literacy levels to prepare for advancement opportunities. 

After determining the proficiency level of employees' abilities to 
follow written instructions, suitable course materials were designed and 
implemented to enhance these skills. The content material included 
routers, move tags and assembly check lists and was designed to relate to 
Product Objective 3: increase ability to follow detailed, sequential 
directions and Product Objective?: increase document literacy related to 
use of manuals, work orders, etc. 

The proficiency level of employees' recognition/knowledge of work- 
related vocabulary and abbreviations was also determined. Suitable 
course materials were then designed and implemented to enhance this 
knowledge. The content material included routers, move tags, assembly 
check lists, and performance reviews and was designed to relate to 
Product Objective 4: increase recognition/ knowledge of workplace 
vocabulary and abbreviations. 

Daily writing exercises were designed to help increase the writing 
skills needed on the job and were intended to relate to Product Objective 
8: increase workplace writing skills. 

The ESL Conversation course was designed to improve 
conversation skills in order to increase social and work related 
communication. By improving verbal, non-verbal, and listening skills 
through role-play and workplace simulations, oral communication skills 
could be enhanced. A class interview of a supervisor and individual verbal 
presentations were used to relate to Product Objective 5: increase com- 
munication skills appropriate to the workplace. 
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VI Evaluation 



Course content evaluations (pre- and post-tests) were administered 
at the beginning of each course content unit, in class, to determine the 
extent of students' mastery of course objectives. The content material 
within each unit pre- and post-test remained unchanged with only slight 
variations in formatting design. 

Students were also asked to evaluate their own English abilities 
both before and after course completion. They were asked to evaluate the 
curriculum content after the first three weeks of class and again at the 
end of the course. This course survey was for students to express their 
interest level in the class content and the relevance of the course 
materials. 

Supervisors were asked to evaluate individual student performance 
at the end of each twelve-hour segment of instruction. This evaluation 
checklist was used to assess the students' noticeable improvements on 
the job. 

Instructors were evaluated by students at the conclusion of each 
course. A checklist was designed to inform the instructors of their 
classroom effectiveness. 

A listening comprehension pre-assessment was administered to the 
employees enrolled in the ESL Conversation class. The assessment 
measure was intended to reflect the possible demands required for these 
employees in the workplace. This instrument was designed to measure 
individual listening comprehension levels and to determine the degree of 
congruity within the class. The assessment consisted of an audio tape of 
three recorded messages with progressive levels of difficulty. Following 
each message, students were required to answer multiple choice 
questions concerning the content of the taped message. The assessment 
took approximately one hour. This assessment was used only at the 
beginning of ihe course as a placement tool. It was not used again at the 
end of the course because other work-related pre-/post-tests were given 
during the course to determine mastery of class materials. 
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Course content evaluations (criterion-referenced pre-/post-tests) 
were given in conjunction with the class lessons. These were 
administered in class at the beginning of two units to determine the 
extent of students' mastery of course objectives. The course content 
materials: polite listening skills, non-verbal communication, interviewing 
skills, and individual student presentations were selected as necessary 
job-related communication tasks. 

Conclusions and Recommendations 

The ESL programs at Natter Manufacturing have been well received 
by management and employees. All parties have expressed satisfaction 
with the curriculum and the students' progress. 

The initial pre-assessment for ESL students was highly skewed 
toward reading abilities. In future ESL programs, a more versatile 
assessment tool designed to also test speaking and listening skills would 
be more beneficial for student placement in ESL classes. Pre- 
assessments should also be done as close to the time of the scheduled 
classes as possible in ^rder to get the most accurate estimates of student 
skills. By waiting up lo ten months to attend class following the initial 
assessment, students' abilities are likely to be different. 
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WORKPLACE LITERACY- FINAL EVALUATION REPORT 



Exhibit B - Conference Brochure 
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Exhibit C ■ Product Objectives Report 



Final Performance Report 
Overview 

This report covers the partnership between Salt Lake Community College (SLCC) and two 
business partners, National Semiconductor Corporation (NSSL) and Natter 
Manufacturing, Inc., to provide opportunities for approximately 225 entry-ievel 
employees to enhance their basic skill levels in order to become more productive^ 
promotable and prepared for technological advances in the industry and to develop a 
model workplace literacy program for dissemination of information, replication at local 
workplaces, and to serve as a basic skills workplace resource for the community. 

The report is divided into the following sections: 1) Performance Report (Product 
Objectives, Schedule of Accomplishments, Quarterly Report {form used for entire 
project}. Changes in Key Personnel, Administrative Issues, Conclusions, 
Recommendations); 2) Dissemination Activities; and 3) Evaluation Activities. 

Performance Report 

The performance report presents data for each product objective. The schedule of 
accomplishments has been submitted in detail in the evaluators' report. A quarterly 
report form is used to detail data for the entire project; it is located at the end of the 
product objectives. 

Some explanation of terminology or procedures is necessary for understanding the data 
presented in the Product Objectives. 

1) 25% attendance: refers to employees who completed at least 25% of the course. This 
figure represents an attempt to be accountable for those employees who participated in 
the program and who should be affected by the protocol. However, it seemed unfair to 
count all employees who registered when some were unable to attend or dropped out after 
only a few days. 

2) Mastery/ Gain Scores: Gain includes all employees who showed any gain in score 
from pre- to post-test; Mastery includes employees who achieved a score of at least 
80% on a post-test; there is some overlap in the data because emptoyees who achieved 
mastery are included in the Gain scores. For example, 20 employees showed gain and 10 
of those same employees achieved mastery. 

3) Hard vs. Soft Data: Hard data is that which resulted from tests or other empirical 
sources. Soft Data is that which resulted from interviews, comments on surveys, etc. 
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Product Objectives 



Product Objective 1: increase the workplace literacy levels of approximately 225 targeted 
employees to prepare them for opportunities to advance to the next level of their job/job family 



Activities conducted to achieve the objective: 

The following classes were conducted: 
A. Natter Manufacturing, Inc. 



SESSION 


CLASS 


DATES 


# SERVED 

^P'^^^- attpnHrinrp^ 


ESL 








Session 1 


Fall ESL 1 
Fall ESL II 
ESL BASIC 


Oct. 1990 - Jan. 1991 
Oct. 1990 - Jan. '1991 
Nov. 1990 - May 1991 


9 
8 
3 


Session 2 


SPRING ESL 1 
SPRING ESL II 


Feb. 1991 - May 1991 
Feb. 1991 - May 1991 


7 
4 


Session 3 


READ/WRITE 
CONVERSATION 


July 1991 - Sept. 1991 
July 1991 - Sept. 1991 


5 
5 


MATH 








Session 1 


REFRESHER 
REFRESHER 


Oct. 1990 - Jan. 1991 
Oct. 1990 - Jan. 1991 


12 
8 


Session 2 


REFRESHER 
ADVANCED 


Feb. 1991 - May 1991 
Feb. 1991 - May 1991 


5 

1 0 


COMMUNICATIONS 






Session 2 


WORKPLACE 


Mar. 1991 - June 1991 


12 


Session 3 


WORKPLACE 


July 1991 - Oct. 1991 


7 


READING & 


WRITING 






Session 3 


TECH READ/WRITE 


July 1991 - Oct. 1991 




Attendance 


Rate Retention 


Rate Total«Enrolled 


Total # Served 

(25%+ attendance 


8 0% 


75% 


1 1 8 


1 02 
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Product Objective 1: increase the workplace literacy levels of approximately 225 targeted 
employees to prepare them for opportunities to advance to the next level of thc-ir job/iob family 



B. National Semiconductor 
PRE-SPC 

Session 1 PRE-SPC Sept. 1990 - Dec. 1990 141 
ESL 

Session 1 ESL 1 Sept. 1990 - Jan. 1991 11 

Session 2 ESL2 Feb. 1991 - June 1991 9 

Sessions ESL 3 July 1991 - Jan. 1992 1 1 
REFRESHER MATH 

Session 2 REFRESHER IMA Feb. 1991 - May 1991 11 

REFRESHER 2MA Feb. 1991 - May 1991 9 

REFRESHER 3MA Feb. 1991 - May 1991 15 

REFRESHER 4MA Feb. 1991 - May 1991 17 

REFRESHER5MA Feb. 1991 - May 1991 16 

REFRESHER6MA Feb. 1991 - May 1991 15 

Sessions REFRESHER 1MB May 1991 - August 1991 15 

REFRESHER 2MB May 1991 - August 1991 6 

REFRESHER 3MB May 1991 - August 1991 8 

REFRESHER 4MB July 1991 -Dec. 1991 8 

REFRESHER 5MB June 1991 - Sept. 1991 9 

REFRESHER 6MB May 1991 - August 1991 8 

READING & WRITING 

Sessions TECH READ/WRITE July 1991 - Nov. 1991 14 
TEAM BUILDING 

Sessions TEAM BUILDING Nov. 1991-Jan. 1992 14 

Attendance Rate Retention Rate Total#Enrolled Total # Served 

(25%+ attendance) 

86% 84% 36 7 34 5 
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Product Objective 1: increase the workplace literacy levels of approximately 225 targeted 
employees to prepare them for opportunities to advance to the next level of their job/job familv 



Explanation of discrepancy if necessary: 

There are some minor differences in the projected data collection in support of this objective 
and the actual data collected. Due to the nationwide economic slump in the computer industry 
beginning Fall 1990 and continuing to the present, both companies experienced partial 
shutdowns, shutdowns, and lay-offs. There was virtually no opportunity for promotion at 
either company. Therefore, data gathered to indicate completion of this objective was garnered 
from surveys administered to supervisors for employees who completed the program. Questions 
in the survey focused on whether the employees had gained enough skill to be considered for or 
given promotions if economic conditions permitted. 

Results: 

Copies of the pre- and post-tests administered to participants and the results are in the 
participants' individual files. 

A. Natter Manufacturing, Inc. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

Refresher Math - Out of 43 enrolled, 35 (81%) attended at least 25% of the course. 
Of the 35, 21 or 60% showed some gain and 1 1 or 26% of those participants showed gain to the 
mastery level. 

English as a Second Language -Teacher observation and anecdotal data indicated that 
seventeen (17) of eighteen (18) students enrolled in three classes in the Fall attended 25% of 
the course. All seventeen (17) improved their skills in varying degrees. In the two Spring 
classes, eleven(ll) of thirteen (13) students completed 25% of the course; of these 11, seven 
(7) achieved mastery, and eight (8) showed some gain. The ESL Conversation class had five (5) 
students of the seven (7) enrolled who completed 25% of the course; two (2) achieved mastery 
and three (3) showed some gain according to pre/post-test results. The ESL Reading/Writing 
class had five (5) students who completed 25% of the course. Hard data was unavailable for 
this class. 



Projected Data Collection 

Individual Employee Files 
Assignments 
Pre/Post Test Results 
Promotion Opportunity Records 
Promotion Records 



Actual Data Collected 

Individual Employee Files 

Assignments 

Pre/Post Test Results 

Supervisor Suiveys 

Training Program Feedback Surveys 
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Product Objective 1: increase the workplace literacy levels of approximately 225 targeted 
employees to prepare them for opportunities to advance to the next level of their job/job family 



In terms of unduplicated participants, 21 of the 23 participants attended at least 25% of 
the course; 16 of those 21, or 76%, either showed some overall gain from pre- to post-test or 
improvement was noted and 7, or 33%, achieved mastery levei on one or more post-tests. 

Workplace Communications -Out of 20 enrolled, 19 (95%) attended at least 25% of 
the course. Of the 19, 14 or 74% showed some gain and 7 or 37% of those participants 
showed gain to the 80% mastery level. 

Technical ReadinqAft/ritinq -Out of 7 enrolled. 7 (100%) attended at least 25% of the 
course. Of the 7. 5 or 71% of participants showed gain to the 80% mastery level. 

Survey Data : 

At the close of each class, a Training Program Fedback survey was administered to 
participants. This evaluation focused on four major areas of the program: General Feedback 
(description and purpose of training); the Trainer (knowledge, teaching ability,); Training 
Materials; Program Design; Other (like/dislike program, relationship of program to job and 
other training opportunities, suggestions for improvement). The items fit one of two forms-a 
rating scale (5=high) or comments/suggestions. Responses indicated that all classes at Natter 
Mfg.. Inc. were rated in the range from 4,27 to 5,0, the average rating for all classes was 4.60. 
The average rating for the program at both companies was 4,26. Class averages are listed 
below. 

Refresher Math: Responses indicated that all classes were rated in the range from 4.57 
to 5.00. the average rating for all classes was 4.73, 

English as a Second Languag e-Responses indicated that all classes were rated in the 
range from 4.40 to 4.91, the average rating for all classes was 4.46, 

Workplace Communications -Responses indicated that all classes were rated in the 
range from 4,27 to 4,31, the average rating for all classes was 4.29, 

Technical ReadinaA/Vritino -Responses indicated that average rating for the class was 

4.46, 

In addition, a Post-Program Participant Survey was administered to participants at the close of 
the program. Demographic data and program data were gathered. The survey focused on 
questions in the following areas: skills acquired through the program that were not present 
before participation; personal goals achieved (promotion, bid on new job, filling out 
accident/sick papers for work, being recognized/appreciated, understanding the job better); 
educational goals achieved (participate in/complete GED courses, enroll in college courses or 
professional training outside of work, increased participation in company training programs); 
increased community involvement (voting, reading newspaper, helping children with home 
work, reading, etc, volunteering in community/school programs); program recommended to cc 
workers; program changes recommended. 
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Product Objective 1: increase the workplace literacy levels of approximately 225 targeted 
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At Natter Mfg. Inc.. 18 responses were returned. The majority of participants stated that the 
program had helped them achieve or work toward the goat of promotion (56%); however, the 
program had not helped them in the area of bidding on a new job. Also, 89% of the respondents 
believed it brought them greater recognition and appreciation. In response to the question about 
program participation helping them understand their job better, 78% of the 18 respondents 
indicated it did (93% of the 15 respondents who answered this question). 

While some respondents indicated they had achieved goals in the area of GED, college enrollment, 
company training programs, etc., the majority of respondents indicated that the program had 
not helped in these areas. The highest positive response occurred in relation to the question 
"Did the program contribute to increased participation in company training programs?"-39% 
of the 18 respondents (47% of the 15 who answered this question) answered "yes." 

Several questions related to community involvement. Sixty-seven percent (67%) of the 
eighteen (18) respondents to this question indicated that as a direct result of the program they 
had increased their involvement in helping their children with reading, homework, and related 
activities. Also, 61% indicated they increased their reading of the newspaper as a result of the 
program. The overwhelming majority of participants indicated they would recommend this 
program to co-workers-78% (88% of the 16 who responded to this statement.). 

Supervisors were also asked to rate participants on a Post-Program Supervisor Rating using a 
4-point scale (4=high). Results, based on 28 responses, are shown in the chart below. NR=No 
Response; Yes or No responses are recorded in the first two columns. 



SCALE 4 3 2 1 NR 

QUESTION Yes No 

As a result of this program, the employee has 

asked about promotion opportunities? 29% 50% 21% 



taken on more responsibility? 32% 39% 11% 21% 

asked about other positions? 1 1 % 4 6 % 21 % 4 % 18 % 

If economic conditions permitted, employee might 
be considered for 

promotion? 25% 36% 11% 4% 25% 

raise? 57% 11% 7% 25% 
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SCALE 

QUESTION 

Do you feel more confident about the employee's 
performance? 

As a result of this program have you noticed any changes 
in the employee's 

behavior? 

job attitude? 

quantity of work? 

quality of work? 

attendance? 

job knowledge? 

Comments are synthesized in the section on Soft Data. 

B. National Semiconductor Corp. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

Refresher Math -- Out of 156 enrolled, 137 (88%) attended at least 25% of the 
course. Of the 137.113 or 82% of the participants showed some gain and 86 or 63% of those 
showed gain to the mastery level. 

Pre-SPC -Out of 141 enrolled, 141 attended at least 25% of the course. Of the 141, 
91 or 65% of the participants showed some gain and 56 or 40% of those showed gain to the 
mastery level. Actually, only 106 participants took both pre- and post-tests. Based on this 
figure. 91 or 86% showed gain and 56 or 53% achieved mastery level. According to company 
records, 98% of the employees who took Pre-SPC achieved SPG certification. 

Fnglifih as a Rprnnd La nquaae -A limited amount of hard data in the form of pre-post test 
results is available for the ESL classes. Out of eleven (11) students enrolled in the ESL 1 class. 
11 (100%) attended 25% of the course. Teacher observation, anecdotal data and evaluation 



4 3 2 1 NR 

Yes No 

39% 36% 4% 21% 

25% 36% 7% 11% 21% 

32% 36% 7% 4% 18% 



32% 39% 7% 21% 

36% 36%> 4% 25% 

39% 29% 7% 25% 

43% 29% 4% 25% 
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indicated that all 1 1 improved their skills in varying degrees. Out of 9 students enrolled in the 
ESL 2 class, 9 (100%) attended 25% of the course. The 9 students showed an average gain of 
13% from pre- to post-test on the objectives tested. Out of eleven (11) students enrolled in 
the ESL 3 class, 11 (100%) attended 25% of the course, in the ESL 3 class. 5 students who 
took the post-tests showed an average gain of 18% for the two units for which pre/post-tests 
wt^ie available. Data that is available suggests that gains were made by participants. 

In terms of unduplicated participants, 28 of the 28 participants attended at least 25% of the 
course; 1 1 of those 28, or 39%, either showed some overall gain from pre- to post-test or 
improvement was noted. Individual data is unavailable for the other 17 participants, but 
aggregate data indicates gain for approximately 13 of those. 

Technical Reading/Writing -Out of 15 enrolled, 14 attended at least 25% of the course. 
Of the 14, 13 or 93% of participants showed some gain and 10 or 71% of those showed gain to 
the mastery level. 

Team Building -Out of 23 enrolled. 22 attended at least 25% of the course. Each 
participant did a pre- and post-self-evaluation and a team evaluation. Fourteen (14) 
participants who actually completed the course reported gains in awareness or knowledge of ten 
teamwork concepts. Supervisors evaluated each team also. Results of self-evaluations and team 
evaluations completed by participants and supervisors yielded results which indicated incn.^ases 
in awareness or knowledge of ten teamwork concepts. The data is presented below. 

Pre-Post Self Evaluation: A self-evaluation form consisting of nine teamwork concepts 
which were to be ranked (5=high) was administered to participants. The total average score 
was 3.52 for the concepts. A post-self evaluation which used the same items asked participants 
to measure the percent of increase in their awareness and/or knowledge of those same concepts. 
Individual self-evaluations showed average increases of 62% to 78% for the nine teamwork 
concepts. The average percent of increase was 70.78 percent. 

Pre-Post Team Evaluation : A team-evaluation form consisting of ten teamwork concepts 
which were to be ranked (5=high) was administered to each participant to evaluate the team. 
The average score was 3.24 for the concepts. A post-self evaluation which used the same items 
asked participants to measure the percent of increase in their awareness and/or knowledge of 
those same concepts. Individual member evaluations of the team showed average increases of 
62% to 77% for the ten teamwork concepts. The average percent of increase was 71.33 
percent. 

Pre-Post Manager Team Evaluation : A team-evaluation form consisting of ten teamwork 
concepts which were to be ranked (5=high) was administered to each manager. The total 
average score was 3.00 for the concepts. A post-self evaluation which used the same items 
asked participants to measure the percent of increase in their awareness and/or knowledge of 
those same concepts. Supervisors' evaluations of the three teams* increases in awareness or 
knowledge of ten teamwork concepts showed average increases of 46%, 67% and 71% for an 
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average increase of 61%. The total average percent of increase was 61.33 percent for the three 
teams. 

Survey Data: 

At the close of each class, a Training Program Fedback form was administered to participants. 
This evaluation focused on four major areas of the program: General Feedback (description and 
purpose of training); the Trainer (knowledge, teaching ability,); Training Materials; Program 
Design; Other (like/dislike program, relationship of program to job and other training 
opportunities, suggestions for improvement). The items fit one of two forms-a rating scale 
(5=high) or comments/suggestions. Responses indicated that all classes at National 
Semiconductor were rated in the range from 3.12 to 4.84, the average rating for all classes was 
4.0, The average rating for the program at both companies was 4.26. Class averages are listed 
below. 

Refresher Math -Responses indicated that all classes were rated in the range from 3,12 
to 4.75, the average rating for all classes was 4.64. 

Pre-SPC -Responses indicated that all classes were rated in the range from 4.06 to 
4.70. the average rating for all classes was 4.38. 

English as a Second Language -Responses indicated that all classes were rated in the 
range from 4.68 to 4.84, the average rating for all classes was 4.76. 

Technical Reading/Writing -Responses indicated that the average rating for the class 
was 4.64. 

Team Building -Responses indicated that the average rating for the class was 4.46 

In addition, a Post-Program Participant Survey was administered to participants at the close of 
the program. Demographic data and program data were gathered- The survey focused on 
questions in the following areas: skills acquired through the program that were not present 
before participation; achieving personal goals (promotion, bid on new job, filling out 
accident/sick papers for work, being recognized/appreciated, understanding the job better); 
achieving educational goals (participate in/complete GED courses, enroll in college courses or 
professional training outside of work, increased participation in company training programs); 
increased community involvement (voting, reading newspaper, helping children with home 
work, reading, etc.. volunteering in community /school programs); recommending program to 
co-workers; recommending program changes. 

At National Semiconductor . 82 responses were returned. The majority of participants stated 
that the program had not helped them achieve goals such as promotion or bidding on a new job 
(74% and 84%. respectively). However. 48% of the respondents believed it brought them 
greater recognition and appreciation. In response to the question about program participation 
helping them understand their job better, 60% of the 82 respondents indicated it did (66% of 
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the 74 respondents who answered this question). While some respondents indicated they had 
achieved goals in the area of GED, college enrollment, company training programs, etc., the 
majority of respondents indicated that the program had not helped in these areas. Given the 
focus of the program, this was not unexpected. The highest positive response occurred in 
relation to the question "Did the program contribute to increased participation in company 
training programs?"- -30% answered "yes." 

Several questions related to community involvement. Thirty-five percent (35%) of the 
seventy-one (71) respondents to this question indicated that as a direct result of the program 
they had increased their involvement in helping their children with reading, homework, and 
related activities. The overwhelming majority of participants indicated they would recommend 
this program to co-workers-83% (91% of the 75 who responded to this statement.). 

Supervisors were also asked to rate participants on a Post-Program Supervisor Rating using a 
4-point scale (4=high). Results, based on 136 responses, are shown in the chart below. 
NR=No Response; Yes or No responses are recorded in the first two columns. 



SCALE 4 3 2 1 NR 

QUESTION Yes No 

As a result of this program, the employee has 

asked about promotion opportunities? 27% 6 7% 6% 

taken on more responsibility? 7% 46% 26% 13% 8% 

asked about other positions? 8 % 2 9 % 2 7 % 28 % 8 % 

If economic conditions permitted, employee might 
be considered for 

promotion? 15% 26% 35% 17% 7% 

raise? 18% 40% 26% 10% 6% 

Do you feel more confident about the employee's 24% 4 0% 25% 7% 4% 

performance? 

As a result of this program have you noticed any changes 
in the employee's 

behavior? 13% 4 6% 22% 13% 6% 
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SCALE 4 


3 


2 


1 


NR 


HON 


Yes 


No 








job attitude? 


13% 


55% 


22% 


4% 


6% 


quantity of work? 


12% 


47% 


35% 


1 % 


5% 


quality of work? 


13% 


50% 


22% 


2% 


13% 


attendance? 


13% 


43% 


28% 


6% 


1 0% 


job knowledge? 


22% 


47% 


19% 


4% 


7% 



Comments are synthesized in the section on Soft Data. 
Discussion of the Results: 

Hard Data : Results indicate that the training resulted in gains in literacy levels for the 
targeted number of participants. Refresher Math results indicated the most significant gains. 
Even though some participants did not achieve the mastery level, examination of the range of 
scores indicates that significant gains were made by some who scored at very low levels on the 
pre-test. Gains were very difficult to measure for ESL participants. Language acquisition 
requires an extensive amount of time. In some instances, data was not collected in an 
appropriate form for subsequent analysis. Aggregate and anecdotal data, however, suggest that 
gains were made by all participants in the ESL component of the program. Team Building was 
conducted as a pilot course and due to the nature of the course, a pre-test of content knowledge 
did not seem an appropriate measure. The measure of success for such a course often comes far 
after the course ends and teams have more opportunity to put into practice the concepts they 
learned. Therefore, the measures used were highly subjective, i.e.. self-evaluations and team 
evaluations by participants and supervisors. These indicated that this was not only a very 
popular course, but the employees rated it very highly in terms of relationship/value to their 
jobs and the new direction the company is pursuing, i.e., self-managed teams. Workplace 
Communications participants showed gains in content knowledge, ability to present to a group; 
they also thought the class was very beneficial. Technical ReadingAfi/riting scores indicated 
gains from pre- to post-test in vocabulary, using work-related manuals, and writing. The Pre 
SPC course was the first one taught and was beset by a number of circumstances which were 
later resolved. Due to these problems only 106 of the 141 participants took both the pre-and 
post-tests. Based on the number who actually took both tests, the results are much more 
favorable. Pre-SPC participants showed gain and 98% went on to achieve SPC Level I 
certification. 
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Survey Data: The majority of participants at Natter Manufacturing. Inc. stated that the 
program had helped them achieve or work toward the goal of promotion (56%); however, the 
program had not helped them in the area of bidding on a new job. This seems to indicate that in 
spite of the economic conditions that prevailed during the grant period, i.e.. the national 
economic slump that greatly affected computer-related industries and the company's shut- 
downs and lay-offs during the grant period, program participants still viewed the program as 
contributing to their chances for promotion even though those opportunities were not 
immediately available. However, 89% of the respondents believed it brought them greater 
recognition and appreciation. In response to the question about program participation helping 
them understand their job better, 78% of the 18 respondents indicated it did (93% of the 15 
respondents who answered this question). This response alone indicates the value of the 
program. Leaders in the field of workplace literacy state that increased job understanding leads 
to increased job productivity, a major goal of workplace literacy programs from the employer's 
perspective. 

While some respondents indicated they had achieved goals in the area of GED, college enrollment, 
company training programs, etc., the majority of respondents indicate that the program had not 
helped in these areas. Given the focus of the program, this was not unexpected. The highest 
positive response occurred in relation to the question "Did the program contribute to increased 
participation in company training programs?""39% of the 18 respondents (47% of the 15 
who answered this question) answered "yes," 

Questions related to community involvement brought about an unexpected outcome of the 
program. Sixty-seven percent (67%) of the respondents to this question indicated that as a 
direct result of the program they had increased their involvement in helping their children 
with reading, homework, and related activities. This has major implications for increases in 
family literacy, and the potential for linking workplace and family literacy programs within 
the workplace. Also, 61% indicated they increased their reading of the newspaper as a result of 
the program. The overwhelming majority of participants indicated the i would recommend this 
program to co-workers— 78% (88% of the 16 who responded to this statement.). This 
indicates the workers valued the program. 

While the majority of participants at National Semiconductor Corp. stated that the program had 
not helped them achieve goals such as promotion or bidding on a new job (74% and 84%, 
respectively), it must be stated that the national economic slump greatly affected computer- 
related industries and the company experienced shut-downs and lay-offs during the grant 
period. Consequently, there was little or no opportunity for promotion or bidding on new jobs 
within the company. In response to the question about program participation helping them 
understand their job better, 60% of the 82 respondents indicated it did (66% of the 74 
respondents who answered this question). This response alone indicates the value of the 
program. Leaders in the field of workplace literacy state that increased job understanding leads 
to increased job productivity, a major goal of workplace literacy programs from the employer's 
perspective. While some respondents indicated they had achieved goals in the area of GED, 
college enrollment, company training programs, etc., the majority of respondents indicate that 
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the program had not helped in these areas. Given the focus of the program, this was not 
unexpected. 

Questions related to community involvement brought about an unexpected outcome of the 
program. Thirty-five percent (35%) of the respondents to this question indicated that as a 
direct result of the program they had increased their involvement in helping their children 
with reading, homework, and related activities. This has major implications for increases in 
family literacy, and the potential for linking workplace and family literacy programs within 
the workplace. The overwhelming majority of panicipants indicated they would recommend 
this program to co-workers-83% (91% of the 75 who responded to this statement.). This is 
a strong indicator of the value workers place on the program. 

Soft Data : Results from survey data were overwhelmingly positive in terms of value 
and relationship of the courses to the employees' jobs. Refresher Math participants reported 
immediate use of skills learned/refreshed and equated it with increases in understanding their 
jobs better and enabling them to assume more responsibility, often on their own initiative. 
They also equalDd it with being able to see and solve problems on their own without assistance. „ 
Workplace Communications and Team Building participants believed that all employees^ should 
take the course and stated that it helped them understand not only their co-workers bettor but 
also management's perspective. They also reported increased ability to communicate with thsir 
co-workers and function better on teams, express themselves better in problem-solving, 
increased understanding of the team concept. Pre-SPC participants indicated a better 
understanding of what SPC was all about and that it made more sense to them. As a result of the 
course, they could see the value of it to the company and the need for accuracy in charting, math, 
etc. ESL participants reported increased understanding of their supervisors, increased 
speakirg skills, and increased understanding of workplace language. Supervisors also reported 
the same kinds of skills increases and in some cases reported increased ability to handle more 
complex responsibilities. They also reported reduced time spent in explaining tasks to 
employees, less need for use of an interpreter, and increased productivity, especially at Natter 
Manufacturing, Inc. Technical Reading/Writing participants reported increases in ability to 
write better, more concise memos, passdowns, and ECN reports and to read and understand 
specifications and work vocabulary. 

Self-confidence and self-esteem were greatly enhanced by program participation as reported by 
participants and supervisors. Most participants indicated that they achieved a sense of being 
recognized and appreciated more c- the job as a result of their participation in the Workplace 
Literacy classes. Supervisors noted that participants tended to ask more questions, work more 
independently, increased job knowledge, increased accuracy in paperwork and math, seek more 
responsibility, help others more often, moved to more difficult jobs, applied new knowledge to 
the job. and in some cases improved attendance as a result of participation in the program. 
Unexpected outcomes occurred in the area of family literacy. Reports of increased involvement 
in children's reading, homework and school activities indicates a benefit from the program that 
was not expected at its inception. This has major implications for increases in family literacy 
and the potential for linking workplace and family literacy programs within the workplace. 
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Overall achievement: 

Overall, the objective to increase the literacy skills of approximately 225 employees to 
prepare them for opportunities to advance to the next level of their job/job family was not only 
met but exceeded. At National Semiconductor Corp . of 273 unduplicated participants who 
attended at least 25% of at least one course, 211 or 77% showed some gain or improvement 
was noted and 139 or 51% achieved mastery level. Eighteen (18) ESL participants were not 
included in the gain figure noted because the data was incomplete; however, anecdotal data and 
teacher observation and the pre/post testing th?: was completed indicate gain for most of them. 
A t Natter Mfg .. of 60 unduplicated participante, who attended at least 25% of at least one course, 
45 or 75% showed some gain or improvement and 23 or 38% achieved mastery level. 

Cc mbined figures for both companies resulted in the following data: There were 333 
uivduplicated participants in the program who attended at least 25% of at least one course. Of 
these 333. 256 or 77% showed some gain and 162 or 49% achieved mastery. If the other 18 
ESL students were included, the gain figure was 274 or 82%. 

Even though there was little or no opportunity for promotion during the project period due to 
national economic conditions which had a heavy impact on these two companies, these emptoyees 
are better prepared and more likely to seek and/or be given opportunities to advance than they 
would have prior to participation in the workplace literacy project. 
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employees so they can obtain additional training for technology upgrades , such as a higher level 
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Activities conducted to achieve the objective: 

The following classes were conducted: 
A. Natter Manufacturing, Inc. 



SESSION 
ESL 

Session 1 

Session 2 
Session 3 

MATH 

Session 1 

Session 2 



CLASS 



DATES 



Fall ESL I 
Fall ESL II 
ESL BASIC 

SPRING ESL I 
SPRING ESL II 

READ/WRITE 
CCNVERSATlCfNj 



REFRESHER 
REFRESHER 

REFRESHER 
ADVANCED 



Oct. 1990 - Jan. 1991 
Oct. 1990 - Jan. 1991 
Nov. 1990 - May 1991 

Feb. 1991 - May 1991 
Feb. 1991 - May 1991 

July 1991 - Sept. 1991 
July 1991 - Sept. 1991 



Oct. 1990 - Jan. 1991 
Oct. 1990 - Jan. 1991 

Feb. 1991 - May 1991 
Feb. 1991 - May 1991 



Mar. 1991 - June 1991 
July 1991 - Oct. 1991 



COMMUNICATIONS 

Session 2 WORKPLACE 
Session 3 WORKPLACE 
READING & WRITING 

Sessions TECH READ/WRITE July 1991 - Oct. 1991 
Attendance Rate Retention Rate Total#Enrolled 



« SERVED 

(25%+ attendance) 



9 
8 
3 

7 
4 

5 
5 



12 
8 

5 
1 0 



1 2 



80% 



75% 



1 1 8 



Total* Served 

(25%+ attendance) 

1 02 
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B. 



National Semiconductor 



PRE-SPC 

Session 1 
ESL 

Sessbn 1 
Sessbn 2 
Sessbn 3 



PRE-SPC 



ESL1 
ESL 2 
ESL 3 



Sept. 1990 - Dec. 1990 



Sept. 1990 - Jan. 1991 
Feb. 1991 - June 1991 
July 1991 - Jan. 1992 



REFRESHER MATH 



Session 1 



Session 3 



141 



1 1 
9 
1 1 



REFRESHER IMA 


Feb. 


1991 


- May 1991 


1 1 


REFRESHER 2f^ 


Feb. 


1991 


- May 1991 


9 


REFRESHER 3MA 


Feb. 


1991 


- May 1991 


1 5 


REFRESHER 4MA 


Feb. 


1991 


- May 1991 


1 7 


REFRESHER 5MA 


Feb. 


1991 


- May 1991 


1 6 


REFRESHER GMA 


Feb. 


1991 


- May 1991 


1 5 


REFRESHER 1MB 


May 


1991 


- August 1991 


1 5 


REFRESHER 2MB 


May 


1991 


- August 1991 


6 


REFRESHER 3MB 


May 


1991 


- August 1991 


8 


REFRESHER 4MB 


July 


1991 


- Dec. 1991 


8 


REFRESHER 5MB 


June 


1991 


- Sept. 1991 


9 


REFRESHER 6MB 


May 


1991 


- August 1991 


8 



READING & WRITING 

Sessbn 3 TECH READ/WRITE July 1991 - Nov. 1991 
TEAM BUILDING 

Sessbn 3 TEAM BUILDING Nov. 1991 - Jan. 1992 



1 4 



1 4 



Attendance Rate Retention Rate TotaltfEnrolled 



86% 



84% 



367 



Totalt Served 

(25%+ attendance) 

345 
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Product Objective 2: increase the workplace literacy levels of approximately 225 
employees so they can obtain additional training for technoloov upgrades , such as a higher level 
of SPC certification. 



Explanation of discrepancy if necessary: 

There are some minor differences in the projected data collection in support of this objective 
and the actual data collected. Due to the nationwide economic slump in the computer industry 
beginning Fall 1990 and continuing to the present, both companies experienced partial 
shutdowns, shutdowns, and lay-offs. Therefore, data gathered to indicate completion of this 
objective was garnered from surveys administered to participants. Questions in the survey 
focused on whether the classes helped participants achieve goals such as participation in other 
training offered by the company. Also, records of SPC certification upgrades or other kinds of " 
training were provided by the training coordinator/personnel manager. 

Results: 

Copies of the pre- and post-tests administered to participants and the results are in the 
participants' individual files. 

A. Natter Manufacturing, Inc. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

Refresher Math - Out of 43 enrolled, 35 (81%) attended at least 25% of the course. 
Of the 35, 21 or 60% of the participants showed some gain and 9 or 26% of those achieved the 
mastery level. 

English as a Second Language -Teacher observation and anecdotal data indicated that 
seventeen (17) of eighteen (18) students enrolled in Ihree classes in the Fall attended 25% of 
the course. All seventeen (17) improved their skills in varying degrees. In the two Spring 
classes, eleven(ll) of thirteen (13) students completed 25% of the course; of these 11, eight 
(8) showed some gain and seven (7) achieved mastery. The ESL Conversation class had five (5) 
students of the seven (7) enrolled who completed 25% of the course; three (3) showed some 
gain according to pre/post-test results and two (2) achieved mastery level. The ESL 
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Projected Data Collection 

Individual Employee Files 

Assignments 

Pre/Post Test Results 

Tech. Upgrade Enrollment Records 

Employee observation on job 



Actual Data Collected 

Individual Employee Files 

Assignments 

Pre/Post Test Results 

Tech. Upgrade Enrollment Records 

Supervisor Surveys 

Training Program Feedback Surveys 

Emplo> ee observation on job 



Product Objective 2: increase the workplace literacy levels of approximately 225 
employees so they can obtain acJditional training for technology upgrades , such as a higher level 
of SPC certification. 



Reading/Writing class had five (5) students who completed 25% of the course. Hard data was 
unavailable for this class. 

In terms of unduplicated participants, 21 of the 23 participants attended at least 25% of 
the course; 16 of those 21, or 76%, either showed some overall gain from pre- to post-test or 
improvement was noted and 7, or 33%, achieved mastery level on one or more post-tests. 

Workplace Communications -Out of 20 enrolled, 19 (95%) attended at least 25% of 
the course. Of the 19, 14 or 74% of the participants showed some gain and 7 or 37% of those 
showed gain to the mastery level. 

Technical Reading/Writing -Out of 7 enrolled, 7 (100%) attended at least 25% of the 
course. Of the 7, 5 or 71% of the participants showed gain to the mastery level. 

B. National Semiconductor Corp. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

Pre-SPC -Out of 141 enrolled, 141 attended at least 25% of the course. Of the 141, 
91 or 65% of the participants showed some gain and 56 or 40% of those showed gain to the 
mastery level. Actually, only 106 participants took both pre- and post-tests. Based on this 
figure, 91 or 86% showed gain and 56 or 53% achieved mastery level. According to company 
records, 98% of the employees who took the Pre-SPC course later achieved SPC Level 1 
certification. 

Refresher Math - Out of 156 enrolled, 137 (88%) attended at least 25% of the 
course. Of the 137, 1 13 or 82% of the participants showed some gain and 86 or 63% of those 
achieved the mastery level. 

English as a Second Language -A limited amount of hard data in the form of pre-post test 
results is available for the ESL classes. Out of eleven (11) students enrolled in the ESL 1 class, 
11 (100%) attended 25% of the course. Teache*- observation, anecdotal data and evaluation 
indicated that all 11 improved their skills in varying degrees. Out of 9 students enrolled in the 
ESL 2 class, 9 (100%) attended 25% of the course. The 9 students showed an average gain of 
13%. Out of eleven (11) students enrolled in the ESL 3 class, 11 (100%) attended 25% of 
the course. In the ESL 3 class, 5 students who took the post-tests showed an average gain of 
18% for the two units for which pre/post-tests were available. Data that is available suggests 
that gains were made by participants. Individual data is unavailable for the other 17 
participants, but aggregate data indicates gain for approximately 13 of those. 
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In terms of unduplicated participants, 28 of the 28 participants attended at least 25% of the 
course; 11 of those 28, or 39%, either showed some overall gain from pre- to post-test or 
improvement was noted. Data was unavailable for seventeen participants. 

Technical Readinq/Writinq -Out of 15 enrolled, 14 attended at least 25% of the course. 
Of the 14, 13 or 93% of the participants showed some gain and 10 or 71% of those achieved 
mastery level. 

Team Building -Out of 23 enrolled, 22 attended at least 25% of the course. Each 
participant did a pre- and post-self-evaiuation and a team evaluation. Fourteen(14) 
participants who actually completed the course reported gains in awareness or knowledge of ten 
teamwork concepts. Supervisors evaluated each team also. Results of self-e*\?;uations and team 
evaluations completed by participants and supervisors yielded results which indicated increases 
in awareness or knowledge of ten teamwork concepts. The results are listed below. 

Pre-Post Self Evaluation: A self-evaluation form consisting of nine teamwork concepts 
which were to be ranked from 1 (low) to 5 (high) was administered to participants. The total 
average score for the concepts was 3,52 , A post-self evaluation which used the same items 
asked participants to measure the percent of increase in their awareness and/or knowledge of 
those same concepts. Individual self-evaluations showed average increases of 62% to 78% for 
the ten teamwork concepts. The average percent of increase was 70.78 percent. 

Pre-Post Team Evaluation : A team-evaluation form consisting of ten teamwork concepts 
which were to be ranked from 1 (low) to 5 (high) was administered to each participant to 
evaluate the team. The average score for the concepts, was 3,24, A post-self evaluation which 
used the same items asked participants to measure the percent of increase in their awareness 
and/or knowledge of those same concepts. Individual member evaluations of the team showed 
average increases of 62% to 77% for the ten teamwork concepts. The average percent of 
increase was 71,33 percent. 

Pre-Post Manager Tmtti Evaluation : A team-evaluation form consisting of nine 
teamwork concepts which were to be ranked from 1 (low) to 5 (high) was administered to each 
manager. The total average score was 3,00 for the concepts. A post-self evaluation which used 
the same items asked participants to measure the percent of increase in their awareness and/or 
knowledge of those same concepts. Supervisors' evaluations of the three teams' increase in 
awareness or knowledge of ten teamwork concepts showed average increases of 46%, 67% and 
71% for an average increase of 61%, The total average percent of increase was 61.33 percent 
for the three teams. 

Survey Data : 

At the ctose of each class, a Training Program Feedback survey was administered to 
participants. This evaluation focused on four major areas of the program: General Feedback 
(description and purpose of training); the Trainer (knowledge, teaching ability,); Training 
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Materials; Program Design; Other (like/dislike program, relationship of program to job and 
other training 

opportunities, suggestions for improvement). The items fit one of two forms- rating scale 
(5=high) or comments/suggestions. Responses indicated that all classes at Natter Mfg .. Inc. 
were rated in the range from 4.27 to 5.0, the average rating for all classes was 4.60. The 
average rating for the program at both companies was 4.26. Class averages are listed below. 

Refresher Math: Responses indicated that all classes were rated in the range from 4.57 
to 5.00; the average rating for all classes was 4.73. 

English as a Second Language -Responses indicated that all classes were rated in the 
range from 4.40 to 4.91; the average rating for all classes was 4.46. 

Technical ReadingA/Vriting -Responses indicated that the average rating for the class 
was 4.46. 

Workplace Communications -Responses indicated that ail classes were rated in the 
range from 4.27 to 4.31; the average rating for all classes was 4,29, 

In addition, a Post-Program Participant Survey was administered to participants at the close of 
the program. Demographic data and program data were gathered. The survey focused on 
questions in the following areas: skills acquired through the program that were not present 
before participation; personal goals achieved (promotion, bid on new job, filling out 
accident/sick papers for work, being recognized/appreciated, understanding the job better); 
educational goals achieved (participate in/complete GED courses, enroll in college courses or 
professional training outside of work, increased participation in company training programs); 
increased community involvement (voting, reading newspaper, helping children with home 
work, reading, etc, volunteering in community/school programs); program recommended to cc 
workers; program changes recommended. 

At Natter Mfg.. Inc.. responses were returned. The majority of participants stated that the 
program h->d helped them achieve or work toward the goal of promotion (56%); however, the 
program had not helped them in the area of bidding on a new job. Also, 89% of the respondents 
believed it brought them greater recognition and appreciation. In response to the question about 
program participation helping them understand their job better, 78% of the 18 respondents 
indicated it did (93% of the 15 respondents who answered this question). 

While some respondents indk^ated they had achieved goals in the area of GED, college enrollment, 
company training programs, etc., the majority of respondents indicate that the program had not 
helped in these areas. The highest positive response occurred in relation to the question "Did 
the program contribute to increased participation in company training programs?""39% of 
the 18 respondents (47% of the 15 who answered this question) answered "yes." 
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Several questions related to community involvement. Sixty-seven percent (67%) of the 
eighteen (18) respondents to this question indicated that as a direct result of the program they 
had increased their involvement in helping their children with reading, homework, ano related 
activities. Also, 61% indicated they increased their reading of the newspaper as a result or the 
program. The overwhelming majority of participants indicated they would recommend this 
program to co-workers-78% (88% of the 16 who responded to this statement,). 

Supervisors were also asked to rate participants on a Post-Program Supervisor Rating using a 
4-point scale (4=high). Results, based on 28 responses, are shown in the chart below. NR=No 
Response; Yes or No responses are recorded in the first two columns. 

SCALE 4 3 2 1 NR 

QUESTION Yes No 

As a result of this program, the employee has 

asked about promotion opportunities? 29% 50% 21% 

taken on more responsibility? 32% 39% 11% 21% 

asked about other positions? 1 1 % 4 6 % 21 % 4 % 18 % 

If economic conditions permitted, employee might 
be considered for 

promotion? 25% 36% 11% 4% 25% 

raise? 57% 11% 7% 25% 

Do you feel more confident about the employee's 39% 36% 4% 21 7o 

performance? 

As a result of thic program have you noticed any changes 
in the employee's 



behavior? 


25% 


36% 


7% 


11% 


21% 


job attitude? 


32% 


36% 


7% 


4% 


18% 


quantity of work? 


32% 


39% 


7% 




21% 


quality of work? 


36% 


36% 


4% 




25% 
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SCALE 



4 



3 



2 



1 



NR 



QUESTION 



Yes 



No 



attendance? 



39% 29% 7% 



25% 



job knowledge? 



43% 29% 4% 



25% 



The same Training Program Fedback survey was administered to participants at National 
Semiconductor . This evaluation focused on four major areas of the program: General Feedback 
(description and purpose of training); the Trainer (knowledge, teaching ability,); Training 
Materials; Program Design; Other (like/dislike program, relationship of program to job and 
other training opportunities, suggestions for improvement). The items fit one of two forms-a 
rating scale (5-high)or comments/suggestions. Responses indicated that all classes at National 
Semiconductor were rated in the range from 3.12 to 4.84, the average rating for all classes was 
4.0. The average rating for the program at both companies was 4.26. Class averages are listed 
below. 

Pre-SPC -Responses indicated that all classes were rated in the range from 4.06 to 
4.70; the average rating for all classes was 4.38. 

Refresher Math -Responses indicated that all classes were rated in the range from 3.12 
to 4.75; the average rating for all classes was 4.64. 

English as a Second Language -Responses indicated that all classes were rated in the 
range from 4.68 to 4.84; the average rating for all classes was 4.76. 

Technical Reading/Writing -Responses indicated that the average rating for the class 
was 4.64. 

Team Building -Responses indicated that the average rating for the class was 4.46 



In addition, a Post-Program Participant Survey was administered to participants at the close of 
the program. Demography data and program data were gathered. The survey focused on 
questions in the following areas: skills acquired through the program that were not present 
before participation; achieving personal goals (promotion, bid on new job, filling out 
accident/sick papers for work, being recognized/appreciated, understanding the job better); 
achieving educational goals (participate in/complete GED courses, enroll in college courses or 
professional training outside of work, increased participation in company training programs); 
increased community involvement (voting, reading newspaper, helping children with home 
work, reading, etc., volunteering in community/school programs); recommending program to • 
co-workers; recommending program changes. 
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At National Semiconductor . 82 responses were retumed. The majority of participants stated 
that the program had not helped them achieve goals such as promotion or bidding on a new job 
(74% and 84%, respectively). However, 48% of the respondents believed it brought them 
greater recognition and appreciation. In response to the question about program participation 
helping them understand their job better, 60% of the 82 respondents indicated it did (66% of 
the 74 respondents who answered this question). While some respondents indicated they had 
achieved goals in the area of GED, college enrollment, company training programs, etc., the 
majority of respondents indicate that the program had not helped in these areas. Given the focus 
of the program, this was not unexpected. The highest positive response occurred in relation to 
the question "Did the program contribute to increased participation in company training 
programs?''-30% answered "yes." 

Several questions related to community involvement. Thirty-five percent (35%) of the 
seventy-one (71) respondents to this question indicated that as a direct result of the program 
they had increased their involvement in helping their children with reading, homework, and 
related activities. The overwhelming majority of participants indicated they would recommend 
this program to co-workers-83% (91% of the 75 who responded to this statement.). 

Supervisors were also asked to rate participants or. a Post-Program Supervisor Rating using a 
4-point scale (4=high). Results, based on 136 responses, are shown in the chart below. 
NR=No Response; Yes or No responses are recorded in the first two columns. 



SCALE 


4 


3 


2 


1 


NR 


QUESTION 


Yes 


No 








As a result of this program, the employee has 












asked about promotion opportunities? 


27% 


67% 






6% 


taken on more responsibility? 


7% 


46% 


2 6% 


13% 


8% 


asked about other positions? 


8% 


29% 


2 7% 


28% 


8% 


If economic conditions permitted, employee might 
be considered for 












promotion? 


15% 


26% 


35% 


17% 


7% 


raise? 


18% 


4 0% 


26% 


1 0% 


6% 


Do you feel more confident about the employee's 


24% 


40% 


25% 


7% 


4% 



performance? 



O V198A00091/SLCC 

ERIC 



.JO 



23 
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SCALE 4 3 2 1 

QUESTION Yes No 



As a result of this program have you noticed any changes 
in the employee's 



behavior? 


1 3% 


4 6% 


22% 


13% 


6% 


job attitude? 


13% 


55% 


22% 


4% 


6% 


quantity of work? 


12% 


47% 


35% 


1 % 


5% 


quality of work? 


13% 


5 0% 


22% 


2% 


1 3% 


attendance? 


1 3% 


43% 


2 8% 


6% 


1 0% 


job knowledge? 


22% 


47% 


19% 


4% 


7% 



Comments for both companies are synthesized in the section on Soft Data. 
Discussion of the Results: 

Hard Data : Results indicate that the training resulted in gains in literacy levels. Math 
results indicate the most significant gains. Even though some participants did not achieve the 
mastery level, examination of the range of scores indicates that some who scored at very low 
levels on the pre-test achieved significant gains even though they did not reach the mastery 
level. Gains were difficult to measure for ESL participants. Where pre*post-test results 
were available, the data indicated that the participants attained mastery in some areas and at 
least gain in others. Anecdotal data and supervisor/teacher observation formed part of the 
evaluation data. Team Building v^as conducted as a pilot course and due to the nature of the 
course, a pre*test of content knowledge did not seem an appropriate measure. The measure of 
success for such a course often comes far after the course ends and teams have more opportunity 
to put into practice the concepts they learned. Therefore, the measures used were highly 
subjective, i.e., self-evaluations and team evaluations by participants and supervisors. These 
indicated that this was not only a very popular course, but the employees rated it very highly in 
terms of relationship/value to their jobs and the new direction the company is pursuing, i.e., 
self-managed teams. Technical Reading/Writing scores indicated gains from pre- to post-test, 
especially in vocabulary, following directions, and writing passdowns, memos, etc.. Pre-SPC 
data indicated significant gains from pre-to post-testing in vocabulary and math skills. 
Actually, only 106 participants took both pre- and post-tests due to difficulties encountered 
during this initial part of the program. Based on this figure, 91 or 86% showed gain and 56 or 
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53% achieved mastery level. At National Semiconductor, 98% of the employees who took Pre- 
SPC later achieved SPC Level 1 certification. 

Soft Data : Results from survey data were overwhelmingly positive in terms of value 
and relationship of the courses to the employees' jobs. Math participants reported immediate 
use of skills learned/refreshed and equated it with increases in understanding their jobs better 
and enabling them to assume more responsibility, often on their own initiative. They also 
equated it with being able to see and solve problems on their own without assistance. Workplace 
Communications and Team Building participants believed that all empk>yees should take the 
course and stated that it helped them understand not only their co-workers better but also 
management's perspective. Pre-SPC participants indicated a better understanding of what SPC 
was all about and that it made more sense to them as a result of the course. They also coukJ see 
the value of it to the company and the need for accuracy in charting, math, etc. ESL participants 
reported increased understanding of their supervisors, increased speaking skills, and increased 
understanding of workplace language. Supervisors of these employees also reported the same 
kinds of skills increases and in some cases reported increased ability to handle more complex 
responsibilities. They also reported reduced time spent in explaining tasks to employees, less 
need for use of an interpreter, and increased productivity, especially at Natter Manufacturing, - 
Inc. Technical Reading/Writing participants reported increases in ability to understand 
specifications, work-related vocabulary, write passdowns, reports and memos, etc. Self* 
confidence and self-esteem were greatly enhanced by program participation as reported by both 
participants and supervisors. Most participants indicated that they achieved a sense of being 
recognized and appreciated more on the job and understanding their jobs better as a result of 
their participation in the Workplace Literacy classes. 

Overall achievement: 

Overall, the objective to increase the workplace literacy levels of approximately 225 
employees so they can obtain additional training for technology upgrades , such as a higher level 
of SPC certification was not only met but exceeded. At National Semiconductor Corp . of 273 
unduplicated participants who attended at least 25% of at least one course, 21 1 or 77% showed 
some gain or improvement was noted and 139 or 51% achieved mastery level. Eighteen (18) 
ESL participants were not included in the gain figure noted because the data was incomplete; 
however, anecdotal data and ceacher observation and the pre/post testing that was completed 
indicate gain for most of them. At Natter Mfg .. of 60 unduplicated participants who attended at 
least 25% of at least one course, 45 or 75% showed some gain or improvement and 23 or 38% 
achieved mastery level. 

Combined figures for both companies resulted in the following data: There were 333 
unduplicated pai(icipants in the program who attended at least 25% of at least one course. Of 
these 333. 256 or 77% showed some gain and 162 or 49% achieved mastery. If the other 18 
ESL students were included, the gain figure was 274 or 82%. 

At National Semiconductor Corp. 98% of participants in Pre-SPC achieved Level 1 SPC 
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certification. All participants upgraded to team status according to the Training Coordinator. 
Although the company originally planned an advanced training course for which the math 
refresher classes were necessary, subsequent conditions changed this plan and the class was not 
offered. It may be offered in the future. 

At Natter Mfg.. Inc. 7 participants in the ESL classes were able to take their re-certification 
test on the computer as all other employees do and achieved re-certification. This experience 
enhanced the employees self-esteem because they could use the computer to test just as afl other 
employees do; prior to this, special arrangements were made for their certification tests. 

Even though there was little or no opportunity for promotion during the project period due to 
national economic conditions which had a heavy impact on these two companies, these employees 
are better prepared and more likely to seek and/or be given opportunities to take advantage of 
technology upgrade training than they would have prior to participation in the workplace 
literacy project. The benefits of the program in regard to technical upgrade training 
opportunities will be further realized as the companies engage in additional training in the 
future. 
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Product Objective 3: increase targeted employees' ability to follow written instructions to 
the 80% proficiency level based on pre-test results 



Activities conducted to achieve the objective: 

The following classes were conducted: 
A. Natter Manufacturing, Inc. 



B. 



SESSION 
ESL 

Session 1 

Session 2 
Session 3 



CLASS 



DATES 



Fall ESL I 
Fall ESL II 
ESL BASIC 

SPRING ESL I 
SPRING ESL II 

READ/WRITE 



Oct. 1990 - Jan. 1991 
Oct. 1990 - Jan. 1991 
Nov. 1990 - May 1991 



Feb. 1991 
Feb. 1991 

July 1991 
July 1991 



May 1991 
May 1991 

Sept. 1991 
Sept. 1991 



READING & WRITING 

Sessions TECH READ/WRITE July 1991 - Oct. 1991 
National Semiconductor 



ESL 

Session 1 
Session 2 
Session 3 



ESL 1 
ESL 2 
ESL 3 



Sept. 1990 - Jan. 1991 
Feb. 1991 - June 1991 
July 1991 - Jan. 1992 



# SERVED 

(25%+ attendance) 



9 
8 
3 

7 
4 

5 
5 



1 1 
9 
1 1 



READING & WRITING 



Sessions TECH READ/WRITE July 1991 - Nov. 1991 



Projected Data Collection 

Pre/Post Test Results 
Simulated Tastes 
Supervisor Feedbacl< re 
transfer to job 



1 4 



Actual Data Collected 

Pre/Post Test Results 
Assignments/simulated tasks 
Supervisor Feedback re 

transfer to job 
Training Program Feedback Surveys 
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Product Objective 3: increase targeted employees' ability to follow written instructions to 
the 80% proficiency level based on pre*test results 



Explanation of discrepancy if necessary: 

There are some minor differences in the projected data collection in support of this objective 
and the actual data collected. Data gathered to indicate completion of this objective was garnered 
from surveys administered to supervisors for employees enrolled in the program. The 
evaluation design included plans to conduct interviews with supervisors at the end of each 12 
hours of instruction. This seemed like a good idea, but in reality was very difficult to do. 
Supervisors felt this was too short a time period to see results. About halfway through the 
project period this was modified to twice during the class session. 

Results: 

Copies of the pre- and post-tests administered to participants and the results are in the 
participants' individual files. 

A. Natter Manufacturing, Inc. 

English as a Second Language -Teacher observation and anecdotal data indicated that 
seventeen (17) of eighteen (18) students enrolled in three classes in the Fall attended 25% of 
the course. All seventeen (17) improved their skills in varying degrees. In the two Spring 
classes. eleven(11) of thirteen (13) students completed 25% of the course; of these 11, eight 
(8) showed some gain and seven (7) achieved mastery. The ESL Conversation class had five (5) 
students of the seven (7) enrolled who completed 25% of the course; three (3) showed some 
gain and two (2) achieved mastery according to pre/post-test results. The ESL Reading/ 
Writing class had five (5) students who completed 25% of the course. Hard data was 
unavailable for this class. 

In terms of unduplicated participants. 21 of the 23 participants attended at least 25% of the 
course; 16 of those 21, or 76%. either showed some overall gain from pre- to post-test or 
improvement was noted and 7, or 33%. achieved mastery level on one or more post-tests. 

In relation to this specific objective for which data is available, seven (33%)participants 
showed gain and six (29%) of those participants achieved mastery. 

Technical ReadinQ/Writina -Out of 7 enrolled, 7 (100%) attended at least 25% of the 
course. Of the 7, 5 or 71% of participants showed gain to the mastery level. 

In relation to this specific objective for which data is available, four (57%) participants 
achieved mastery. 

B. National Semiconductor Corp. 

Individual Employee Files/Assignments: 
On file. 
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Pre/Post Tests: 

English as a Second Language ^^A limited amount of hard data in the form of pre-post 
test results is available for the ESL classes. Out of eleven (11) students enrolled in the ESL 1 
class, 11 (100%) attended 25% of the course. Teacher observation, anecdotal data and 
evaluation indicated that all 11 improved their skills in varying degrees. Out of 9 students 
enrolled in the ESL 2 class, 9 (100%) attended 25% of the course. The 9 students showed an 
average gain of 13% on the pre- to post-tests for the objectives tested. Out of eleven (11) 
students enrolled in the ESL 3 class. 11 (100%) attended 25% of the course. In the ESL 3 
class, 5 students who took the post-tests showed an average gain of 18% for the two units for 
which pre/post-tests were available. Data that is available suggests that gains were made by 
participants. 

In terms of unduplicated participants, 28 of the 28 participants attended at least 25% of the 
course; 1 1 of those 28, or 39%, either showed some overall gain from pre- to post-test or 
improvement was noted. 

Although testing was conducted that related to this objective, individual pre/post-test scores 
were not available to document achievement of this particular objective. However, seven (7) 
participants achieved an average increase of 24% in their scores from pre- to post-test. 

Technical Reading/Writing -Out of 15 enrolled, 14 attended at least 25% of the course. 
Of the 14, 13 or 93% showed some gain and of those, 10 or 71% of participants showed gain to 
the mastery level . 

In relation to this specific objective for which data is available, eleven (11)or 79% of the 
participants showed gain and eight (8) or 67% of those achieved mastery. 

Survey Data: 

At the close of each class, a Training Program Fedback survey was administered to participants. 
This evaluation focused on four major areas of the program: General Feedback (description and 
purpose of training); the Trainer (knowledge, teaching ability,); Training Materials; Program 
Design; Other (like/dislike program, relationship of program to job and other training 
opportunities, suggestions for improvement). The items fit one of two forms- rating scale 
(5=high) or comments/suggestions. Responses indicated that all classes a t Natter Mfg .. Inc. 
were rated in the range from 4.27 to 5.0, the average rating for all classes was 4.60. The 
average rating for the program at both companies was 4.26. Class averages are listed below. 

Refresher Math: Responses indicated that all classes were rated in the range from 4.57 
to 5.00; the average rating for all classes was 4.73. 

En glish as a Second Language -Responses indicated that all classes were rated in the 
range from 4.40 to 4.91; the average rating for all classes was 4.46. 
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Technical ReadingAA/riting -Responses indicated that the average rating for the class 
was 4.46. 

Workplace Communications -Responses indicated that all classes were rated in the 
range from 4.27 to 4.31; the average rating for ail classes was 4.29. 

The same Training Program Fedback survey was administered to participants at National 
Semiconductor . This evaluation focused on four major areas of the program: General Feedback 
(description and purpose of training); the Trainer (knowledge, teaching ability,); Training 
Materials; Program Design; Other (like/dislike program, relationship of program to job and 
other training opportunities, suggestions for improvement). The items fit one of two forms-a 
rating scale (5=high) or comments/suggestions. Responses indicated that all classes at National 
Semiconductor were rated in the range from 3.12 to 4.84. the average rating for all classes was 
4.0. The average rating for the program at both companies was 4.26. Class averages are listed 
below. 

Pre-SPC -Responses indicated that all classes were rated in the range from 4.06 to 
4.70; the average rating for all classes was 4.38. 

Refresher Math -ResDonses indicated that all classes were rated in the range from 3.12 
to 4.75; the average rating for all classes was 4.64. 

English as a Second Language -Responses indicated that all classes were rated in the 
range from 4.68 to 4.84; the average rating for all classes was 4.76. 

Technical Readinq/Writing -Responses indicated that the average rating for the class 
was 4.64. 

Team Building -Responses indicated that the average rating for the class was 4.46. 
Discussion of the Results: 

Hard Data : The ESL start-up classes at both companies were truly experimental. Teachers and 
companies new to workplace literacy worked hard to determine needs and to design a curriculum 
that would be beneficial to employees. However, as in any new venture, a great deal was learned 
from the initial attempts. Although attempts were made to evaluate students, it was not in a 
formal, criterion-referenced style. No pre/post unit tests were given in the first round of ESL 
classes at either company. A pre-test designed by the instructors was heavily skewed to reading 
skills. The initial post-test was a modified version of the pre-test and heavy reliance was 
placed on teacher observation and anecdotal data for evaluation. 

After the first round of ESL classes, instructors attempted to design curriculum that was more 
functional-context oriented and centered around units. Criterion-referenced tests were 
designed for the units. This enabled instructors to measure and collect data that was more 
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useful. However, it is still difficult to measure gains in ESL classes. Language acquisition is a 
complicated and lengthy process. 

Another factor which should be considered in evaluating the data centers on the economic 
conditions the companies were experiencing from the fall of 1990 to the completion of the 
program in 1992. A severe economic slump in the national economy and computer-related 
industry had a significant impact on these companies and their employees. Employees were 
experiencing shut-downs and lay-offs throughout this time and the psychological environment 
this created was not conducive to learning. Employees are to be commended for their leaning 
gains and companies for their commitment to the program in the face of hard times. 

Soft Data : Results from survey data were overwhelmingly positive in terms of value and 
relationship of the courses to the employees' jobs. ESL participants reported increased 
understanding of their supervisors, increased speaking skills, and increased understanding of 
workplace language and their jobs. Supervisors also reported the same kinds of skills increases 
and in some cases reported increased ability to hand.: more complex responsibilities. They 
also reported reduced time spent in explaining tasks to employees, less need for use of an 
interpreter, and increased productivity, especially at Natter Manufacturing, Inc. Technical 
ReadingA^riting participants reported increases in understanding workplace vocabulary and 
specifications. Managers reported seeing changes in both the number and quality of memos, 
passdowns, etc. written by employees who participated in the program. Self-confidence and sel 
esteem were greatly enhanced by program participation as reported by participants and 
supervisors. 

Overall achievement: 

Overall, the objective was partially met and relies heavily on teacher observation and anecdotal 
data for ESL. Both companies had a total of 51 participants in ESL who attended at least 25% of 
the course; this is unduplicated headcount. At Natter Mfg., of 21 participants who attended at 
least 25% of the course, 16 or 76%, either showed some overall gain from pre- to post-test or 
improvement was noted and 7 or 33% achieved mastery level on one or more post-tests. 
Specifically, 7 of 11 participants with pre/post-test scores(64%) showed gain on this 
objective and 6 (55%) achieved mastery. At National Semiconductor Corp., of 28 unduplicated 
participants enrolled in ESL, 1 1 or 39% showed some improvement. Hard data was unavailable 
for individuals, but pre-to post-test average scores indicate gain for approximately 7 others. 

Together the companies had a total of 21 participanls in Technical Reading/Writing who 
attended at least 25% of the course. Eighteen (18). or 86%, showed some overall gain and 15, 
or 71%, of those achieved mastery level. At Natter Mfg .. of 7 participants who attended at least 
25% of the course. 5 or 71% showed overall gam and achieved mastery level. Specifically, 4 
of 7 participants with pre/post-test scores (57%) showed gain on this objective. At National 
Semiconductor Corp .. 14 participants attended at least 25% of the course; of that 14, 13 or 
93% showed some overall gain and 10 or 71% of those achieved mastery level. Specifically, 
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Product Objective 3: increase targeted employees' ability to follow written instructions to 
the 80% proficiency level based on pre-test results. 



11 of 14 participants with pre/post-test scores (79%) showed gain on this objective and 8 of 
those (57%) achieved mastery. 

Of the targeted 69 unduplicated participants for this objective, the majority. 45 (65%), 
showed overall gain or improvement was noted. Specifically, there are pre/post-test records 
for 32 participants. Of those, 22 of the 32 participants (69%) showed gain on this objective 
and 14 (44%)of those achieved mastery. 
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Product Objective 4: increase targeted employees' recognition/knowledge of work-related 
vocabulary to the 80% proficiency level based on pre-test results. 



Activities conducted to achieve the objective: 

The following classes were conducted: 
A. Natter Manufacturing, Inc. 



SESSION CLASS DATES # SERVED 

(25%+ attendance) 

ESL 

Session 1 Fall ESL I Oct. 1990 - Jan. 1991 9 

Fall ESL 11 Oct. 1990 - Jan. 1991 8 

ESL BASIC Nov. 1990 - May 1991 3 

Session 2 SPRING ESL I Feb. 1991 - May 1991 7 

SPRING ESL II Feb. 1991 - May 1991 4 

Sessions READ/WRITE July 1991 - Sept. 1991 5 

CCNVERSATX3N July 1991 - Sept. 1991 5 

READING & WRITING 

Sessions TECH READ/WRITE July 1991 - Oct. 1991 7 
B. National Semiconductor 
PRE-SPC 

Session 1 PRE-SPC Sept. 1990 - Dec. 1990 141 
ESL 

Sessran 1 ESL 1 Sept. 1990 - Jan. 1991 1 1 

Sess»n2 ESL2 Feb. 1991 - June 1991 9 

SessKinS ESL 3 July 1991 - Jan. 1992 1 1 

READING & WRITING 

SesskjnS TECH READA/VRITE July 1991 - Nov. 1991 14 
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Product Objective 4: increase targeted employees* recognition/knowledge of work-related 
vocabulary to the 80% proficiency level based on pre-test results. 



Projected Data Collection 



Actual Data Collected 

Pre/Post Test Results 
Assignments/simulated tasks 
Supervisor Feedback re 



Pre/Post Test Results 
Simulated Tasks 



Supervisor Feedback re 



transfer to job 



transfer to job 
Training Program Feedback Surveys 



Explanation of discrepancy if necessary: 

There are some minor differences in the projected data collection in support of this objective 
and the actual data collected. Data gathered to indicate completion of this objective was garnered 
from surveys administered to employees enrolled in the program and to their supervisors. The 
evaluation design included a meeting between instructors and supervisors of enrollees at the end 
of each 12 hour period of instruction. The purpose of this meeting was to determine transfer of 
skills taught to the job. In reality, meeting this often was problematic because the supervisors 
usually did not see the transfer of skill that rapidly and they also believed the meetings 
unnecessary. Also, it was sometimes difficult to set up a meeting with supervisors for some 
time, so one 12 hour period would end and another be partially completed before the meeting 
occurred for the first time. For these reasons, the instructors and supervisors met twice- 
halfway through the course and at the end of the course. 

Results: 

Copit^ of the pre- and post-tests administered to participants and the results are in the 
participants' individual files. 

A. Natter Manufacturing, Inc. 

English as a Second Lanquaqe -Teacher observation and anecdotal data indicated that 
seventeen (17) of eighteen (18) students enrolled in three classes in the Fall attended 25% of 
the course. All seventeen (17) improved their skills in varying degrees. In the two Spring 
classes, eleven(ll) of thirteen (13) students completed 25% of the course; of these 11. eight 
(8) showed some gain and. of those, seven (7) achieved mastery. The ESL Conversation class 
had five (5) students of the seven (7) enrolled who completed 25% of the course; three (3) 
showed some gain according to pre/post-test results and two (2) of those achieved mastery. The 
ESL ReadingAWriting class had five (5) students who completed 25% of the course. Hard data 
was unavailable for this class. 

In terms of unduplicated participants. 21 of the 23 participants attended at least 25% of the 
course; 15 of those 21. or 71%. either showed some overall gain from pre- to post-test or 
improvement was noted and 7. or 33%. achieved mastery level on one or more post-tests. 



O V198A00091/SLCC 

EHJC 



34 



Product Objective 4: increase targeted employees' recognition/knowledge of work-related 
vocabulary to the 80% proficiency level based on pre-test results. 



In relation to this specific objective for which data is available, seven (33%)participants 
showed gain and six (29%) of those achieved mastery. 

Technical Rgading/Wrlting-Out of 7 enrolled, 7 (100%) attended at least 25% of the course. 
Of the 7. 5 or 71% of participants showed gain to the mastery level . 

In relation to this specific objective for which data is available, four (57%) participants 
achieved mastery. 

B. National Semiconductor Corp. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

Pre-SPC-Out of 141 enrolled, 141 attended at least 25% of the course. Of the 141. 
91 or 65% of the participants showed some gain and 56 or 40% of those showed gain to the 
mastery level. Actually, only 106 participants took both pre- and post-tests. Based on this 
figure, 91 or 86% showed gain and 56 or 53% achieved mastery level. According to company 
records, 98% of the employees who took the Pre-SPC course later achieved SPC Level 1 
certification. 

English a s a Second Language -A limited amount of hard data in the form of pre-post 
test results is available for the ESL classes. Out of eleven (11) students enrolled in the ESL 1 
class. 11 (100%) attended 25% of the course. Teacher observation, anecdotal data and 
evaluation indicated that all 1 1 improved their skills in varying degrees. Out of 9 students 
enrolled in the ESL 2 class, 9 (100%) attended 25% of the course. The 9 students showed an 
average gain of 13% on the post-test for the objectives tested. Out of eleven (11) students 
enrolled in the ESL 3 class, 11 (100%) attended 25% of the course. In the ESL 3 class. 5 
students who took the post-tests showed an average gain of 18% for the two units for which 
pre/post-tests were available. Data that is available suggests that gains were made by 
participants. 

Althouch testing was conducted that related to this objective, individual pre/post-test scores 
were not available to document achievement of this particular objective. However, the average 
score decreased for the nine participants who took the pre- and post-test. 

Technical ReadinqAft/riting -Out of 15 enrolled. 14 attended at least 25% of the course. 
Of the 14, 13 or 93% of the participants showed some gain and 10 or 71% of those achieved 
the mastery level. 

In relation to this specific objective for which data is available, thirteen (13) or 93% of 
participants showed gain and eight (8) or 67% of those achieved mastery. 
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Product Oblectlve 4: increase targeted employees' recognition/knowledge of work-related 
vocabulary to the 80% proficiency level based on pre-test results. 



Survey Data: 

At the close of each class, a Training Program Fedback survey was administered to 
participants. This evaluation focused on four major areas of the program: General Feedback 
(description and purpose of training); the Trainer (knowledge, teaching ability,); Training 
Materials; Program Design; Other (like/dislike program, relationship of program to job and 

other training opportunities, suggestions for improvement). The items fit one of two forms-a 
rating scale (5=high) or comments/suggestions. Class averages are listed below. 

A. Natter Manufacturing, Inc. 

English as a Second Language -Responses indicated that all classes were rated in the 
range from 4.40 to 4.91; the average rating for all classes was 4.46. 

Technical ReadinqA^riting -Responses indicated that the average rating for the class 
was 4.46. 

B. National Semiconductor Corp. 

Pre^SPC -Responses indicated that all classes were rated in the range from 4.06 to 
4.70; the average rating for all classes was 4.38. 

English as a Second Language -Responses indicated that all classes were rated in the 
range from 4.68 to 4.84; the average rating for all classes was 4.76. 

Technical Reading/Writing -Responses indicated Xh^i the average rating for the class 
was 4.64. 

Discussion of the Results: 

Hard Data : The ESL start-up classes at both companies were truly experimental. 
Teachers and companies new to workplace literacy worked hard to detenmine needs and to design 
a cumculum that would be beneficial to employees. However, as in any new venture, a great 
deal was learned from the initial attempts. Although attempts were made to evaluate students, it 
was not in a formal, criterion-referenced style. No pre/post unit tests were given in the first 
round of ESL classes at either company. A pre-test designed by the instructors was heavily 
skewed to reading skills. The initial post-test was a modified version of the pre-test and heavy 
reliance was placed on teacher observation and anecdotal data for evaluation. 

After the first round of ESL classes, instructors attempted to design curriculum that was more 
functional-context oriented and centered around units. Criterion-referenced tests were 
designed for the units. This enabled instructors to measure and collect data that was mc«'e 
useful. However, it is still difficult to measure gains in ESL classes. ESL language skill 
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Product Objective 4: increase targeted employees' recognition/knowledge of work-related 
vocabulary to the 80% proficiency level based on pre-test results. 



development is a complicated and lengthy process. 

Other factors which should be considered in evaluating the data center on the economic 
conditions the companies were experiencing from the fall of 1990 to the completion of the 
program in 1992. A severe economic slump in the national economy and the computer industry 
had a significant impact on these companies and their employees. Employees were experiencing 
shut-downs and layoffs throughout this time and the psychological environment 
this creates is not conducive to learning. Employees are to be commended for their learning 
gains and companies for their commitment to the program in the face of hard times. 

The Pre-SPC workshops were the first ones taught in the program and, as a result, experienced 
all the difficulties encountered in establishing a new program. Consequently, not all 
participants completed both pre- and post-tests. Of the 141 who completed 25% of the course, 
only 106 participants took both tests. Based on this data, gains were significant and mastery 
was achieved by over half of those who completed testing. 

Even though the Technical Reading/Writing class at National Semiconductor was interrupted for 
approximately two months due to a "soft-shutdown," participants still showed learning gains. 
This may be associated with the functional context curriculum allowing them to use their new 
knowledge on the job so retention was higher than might be expected in more traditional Adult 
Basic Education classes. 

Soft Data : Results from survey data were overwhelmingly positive in terms of value 
and relationship of the courses to the employees' jobs. ESL participants reported increased 
understanding of their supervisors, increased speaking skills, and increased understanding of 
workplace language and their jobs. Supervisors also reported the same kinds of skills increases 
and in some cases reported increased ability to handle more complex responsibilities, reduced 
time spent in explaining tasks to employees, less need for use of an interpreter, and increased 
productivity. Pre-SPC participants reported significant gains in their understanding of the 
math and technical language used in the SPC field. They also reported greater understanding of 
the value of SPC to the company and their role in meeting the standards. Technical 
Reading/Writing participants reported increases in workplace vocabulary, ability to write 
memos, passdowns, and reports and understanding of the job. Self-confkience and self-esteem 
were greatly enhanced by program participation as reported by participants and supervisors. 

Overall achievement: 

Overall, the objective was partially met and relies heavily on teacher observation and anecdotal 
data for ESL. Of the 198 targeted unduplicated partrcipants who attended at least 25% of at least 
one course for this objective, 132 (67%) showed gain and 76 (38%) achieved mastery level. 

Both companies had a total of 51 unduplicated participants in ESL who attended at least 25% of 
the course. At Natter Mfg .. of 21 participants who attended at least 25% of the course, 16 or • 
76% either showed some gain from pre- to post-test or improvement was noted and 7 or 33% 
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Product Objective 4: increase targeted employees* recognition/knowledge of work-related 
vocabulary to the 80% proficiency level based on pre-test results. 



achieved mastery level. At National Semiconductor Corp. . of 31 participants enrolled in ESL, 
28 unduplicated participants attended 25% of the course. Gain or improvement was noted for 
1 1 of those; individual data was incomplete for the balance of the participants but aggregate 
data indicated gain for an additional eight students. 

Together the companies had a total of 21 participants in Technical Reading/Writing who 
attended at least 25% of the course. Of that total number, 18 or 86% showed some gain and of 
those 15 or 71% achieved mastery level. At Natter Mfg .. of 7 participants who attended at 
least 25% of the course, 5 or 71% showed gain to the mastery level. At National 
Semiconductor Corp. . 14 participants attended at least 25% of the course; of that 14, 13 or 
93% showed some gain and 10 or 71% of those achieved mastery level. 

Out of 141 enrolled in the Pre-SPC workshops, 141 attended at least 25% of the course. Of the 
141, 91 or 65% of the participants showed some gain and 56 or 40% of those showed gain to 
the mastery level. Actually, only 106 participants took both pre- and post-tests. Based on 
this figure. 91 or 86% showed gain and 56 or 53% achieved mastery level. According to 
company records, 98% of the employees who took the Pre-SPC course later achieved SPC Level 
1 certification. 
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Product Objective S: increase communications skills appropriate to the workplace 

Activities conducted to achieve the objective: 

The following classes were conducted: 

A. Natter Manufacturing, Inc. 

SESSION CLASS DATES # SERVED 

(25%+ attendance) 

ESL 

Session 1 Fall ESL I Oct. 1990 - Jan. 1991 9 

Fall ESL II Oct. 1990 - Jan. 1991 8 

ESL BASIC Nov. 1990 - May 1991 3 

Session 2 SPRING ESL I Feb. 1991 - May 1991 7 

SPRING ESL II Feb. 1991 - May 1991 4 

Sessions READ/WRITE July 1991 - Sept. 1991 5 

CXDNVERSATION July 1991 - Sept. 1991 5 

READING & WRITING 

Sessions TECH READA/VRITE July 1991 - Oct. 1991 7 
COMMUNICATIONS 

Sessbn2 WORKPUkCE Mar. 1991 - June 1991 12 

Sessions WORKPLACE July 1991 - Oct. 1991 7 

B. National Semiconductor 
ESL 

Sessbn 1 ESL 1 Sept. 1990 - Jan. 1991 11 

Session 2 ESL 2 Feb. 1991 - June 1991 9 

Sessions ESL3 July 1991 - Jan. 1992 1 1 

READING & WRITING 

Sessions TECH READA/VRITE July 1991 - Nov. 1991 14 
TEAM BUILDING 

Sessions TEAM BUILDING Nov. 1991-Jan. 1992 22 
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Projected Data Collection 

Individual Employee Files 
Assignments 
Pre/Post Test Results 
Observation/monitoring of 

skill transfer to job/team meetings 
Monitor written communications for 
accuracy 



Actual Data Collected 

Individual Employee Files 

Assignments 

Pre/Post Test Results 

Supervisor Surveys 

Training Program Feedback Surveys 



Explanation of discrepancy if necessary: 



There are some minor differences in the projected data collection in support of this objective 
and the actual data collected. Data gathered to indicate completion of this objective was garnered 
from surveys administered to employees enrolled in the program and their supervisors. The 
evaluation design included a meeting between instructors and supervisors of enrollees at the end 
of each 12 hour period of instruction. The purpose of this meeting was to determine transfer of 
skills taught to the job. In reality, meeting this often was problematic because the supervisors 
usually did not see the transfer of skill that rapidly and they also believed the meetings 
unnecessary. Also, it was sometimes difficult to set up a meeting with supervisors for some 
time, so one 12 hour period would end and another be partially completed before the meeting 
occurred for the first time. For these reasons, the instructors and supervisors met twice- 
halfway through the course and at the end of the course. Also, monitoring written 
communications was not practical. Simulated task assignments were substituted. 



Results: 



A. Natter Manufacturing, Inc. 

English as a Second Language- Teacher observation and anecdotal data indicated that 
seventeen (17) of eighteen (18) students enrolled in three classes in the Fall attended 25% of 
the ccurse. All seventeen (17) improved their skills in varying degrees. In the two Spring 
classes, eleven(ll) of thirteen (13) students completed 25% of the course; of these 11. eight 
(8) showed some gain and seven (7) of those achieved mastery. The ESL Conversation class had 
five (5) students of the seven (7) enrolled who completed 25% of the course; three (3) showed 
some gain according to pre/post-test results and two (2) achieved mastery . The ESL 
Reading/Writing class had five (5) students who completed 25% of the course. Hard data was 
unavailable for this class. 

In terms of unduplicated participants, 21 of the 23 participants attended at least 25% of the 
course; 16 of those 21. or 76%, either showed some overall gain from pre- to post-test or 
improvement was noted and 7 or 33% achieved mastery level on one or more post-tests. 

Anecdotal data, teacher/supervisor observation, and simulated tasks indicated improvement for 
the majority of the students. 
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Product Objective 5: increase communications skills appropriate to the workplace. 



Technical Readina/Writing -Out of 7 enrolled. ^ (ioo%) attended at least 25% of the course. 
Of the 7, 5 or 71% of participants showed gain to the mastery level . 

In relation to this specific objective for which data is available, four (4) participants achieved 
mastery. 

Workplace Communications -Out of 20 enrolled. 19 (95%) attended at least 25% of the 
course. Of the 19, 14 or 74% showed some gain and 7 or 37% of those showed gain to the 
mastery level. 

B. National Semiconductor, Corp. 

English as a Second Language -A limited amount of hard data in the form of pre-post 
test results is available for the ESL classes. Gut of eleven (11) students enrolled in the ESL 1 
class, 11 (100%) attended 25% of the course. Teacher observation, anecdotal data and 
evaluation indicated that all 1 1 improved their skills in varying degrees. Out of 9 students 
enrolled in the ESL 2 class, 9 (100%) attended 25% of the course. The 9 students showed an 
average gain of 13%. Out of eleven (11) students enrolled in the ESL 3 class, 11 (100%) 
attended 25% of the course. In the ESL 3 class. 5 students who took the post-tests showed an 
average gain of 18% for the two units for which pre/post -tests were available. Data that is 
available suggests that gains were made by participants. 

The average score for five students increased 23% from pre- to post-test for this particular 
objective. No other data was available. 

Technical ReadingAA/riting -Out of 15 enrolled, 14 attended at least 25% of the course. 
Of the 14. 13 or 93% of the participants showed some gain and 10 or 71% of those showed gain 
to the mastery level. 

Team Building -Out of 23 enrolled, 22 attended at least 25% of the course. Each 
participant did a pre and post self-evaluation and a team evaluation. Fourteen(14) participants 
who actually completed the course reported gains in awareness or knowledge of ten teamwork 
concepts. Supervisors evaluated each team also. Results of self-evaluations and team 
evaluations completed by participants and supen/isors yielded results which indicated increases 
in awareness or knowledge of ten teamwork concepts. 

Pre-Post Self Evaluation: 

A self-evaluation form consisting of nine teamwork concepts which were to be ranked 
from 1 (low) to 5 (high) was administered to participants. The total average score was 3.52 
for the concepts. A post-self evaluation which used the same items asked participants to 
measure the percent of increase in their awareness and/or knowledge of those same concepts. 
Individual self-evaluations showed average increases of 62% to 78% for the ten teamwork 
concepts. The average percent of increase was 70.78 percent. 



V198A00091/SLCC 



f O 



41 
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Pre-Post Team Evaluation: 

A team-evaluation form consisting of ten teamwork concepts which were to be ranked 
from 1 (low) to 5 (high) was administered to each participant to evaluate his/her team. The 
average score was 3.24 for the concepts. A post-self evaluation which used the same items 
asked participants to measure the percent of increase in their awareness and/or knowledge of 
those same concepts. Individual member evaluations of the team showed average increases of 
62% to 77% for the ten teamwork concepts. The average percent of increase was 71.33 
percent. 

Pre-Post Manager's Team Evaluation: 

A team-evaluation form consisting of ten teamwork concepts which were to be ranked 
from 1 (low) to 5 (high) was administered to each manager. The total average score was 3.00 
for the concepts. A post-self evaluation which used the same items asked participants to 
measure the percent of increase in their awareness and/or knowledge of those same concepts. 
Supervisors' evaluations of the three teams' increase in awareness or knowledge of ten 
teamwork concepts showed average increases of 46%, 67% and 71% for an average increase of 
61%. The total average percent of increase was 61.33 percent for the three teams. 

Discussion of the Results: 

Hard Data : Results indicate that the training resulted in gains in communication skills. 
Gains were very difficult to measure for ESL participants and initially relied heavily on 
anecdotal data and teacher/supervisor observation and evaluation of improvement. Team 
Building was conducted as a pitot course and due to the nature of the course, a pre-test of content 
knowledge did not seem an appropriate measure. The measure of success for such a course often 
comes far after the course ends and teams have more opportunity to put into practice the 
concepts they learned. Therefore, the measures used were highly subjective, i.e., self- 
evaluations and team evaluations by participants and supervisors. These indicated that this was 
not only a very popular course, but the employees rated it very highly in terms of its 
relationship/value to their jobs and the new direction the company is pursuing, i.e., self- 
managed teams. They reported increases for themselves and their teams in ability to work as a 
team, express their point of view, handle conflict, etc. Workplace Communications participants 
showed gains in content knowledge and ability to make presentations to teams and/or 
management groups. Technical Reading/Writing participants showed gains in both vocabulary 
and written communications skills related to such on-the-job activities such as writing 
passdowns, ECN changes, and memos. Although not specifically tested, it is anticipated that this 
carries over to other forms of written communication on the job. 

Soft Data : Results from survey data were positive in terms of value and relationship of 
the courses to the employees' jobs. Workplace Communications and Team Building participants 
believed that all emptoyees should take the course and stated that it helped them understand not 
only their co-workers better but also management's perspective. They also reported increased 
ability to communicate with their co-workers and function better on teams, express themselves 
better in problem*solving, understand the team concept, and speak up in meetings, etc. ESL 
participants reported increased understanding of their supervisors, increased speaking skills, 
and increased understanding of workplace language. Supervisors of ESL employees also 
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Product Objective 5: increase communications skills appropriate to the workplace 



reported the same kinds of skills increases and in some cases reported increased ability to 
handle more complex responsibilities. In addition they reported reduced time spent in 
explaining tasks to employees, less need for use of an interpreter, and increased productivity, 
especially at Natter Manufacturing, inc.. Self-confidence and self-esteem were greatly 
enhanced by program participation as reported by participants and supervisors. Supervisors 
noted that participants tended to ask more questions, work more independently, show increased 
job knowledge, seek more responsibility, help others more often, move to more difficult jobs, 
and apply new knowledge to the job. 

Overall achievement: 

Overall, the objective was partially met and relied heavily on teacher observation and 
anecdotal data for ESL. Both companies had a total of 51 unduplicated participants in ESL who 
attended at least 25% of the course. At Natter f^fg.. of 21 participants who attended at least 
25% of the course. 16 or 76% either showed some gain from pre- to post-test or 
improvement was noted and 7 or 33%% achieved mastery level. At National Semiconductor 
Corp., of 28 unduplicated participants enrolled in ESL, data that was available indicated gain for 
11; data for other individuals was incomplete. However, aggregate data indicated gain for 
approximately eight others. 

Together the companies had a total of 21 participants in Technical Reading/Writing who 
attended at least 25% of the course. Of that total number, 18 or 86% showed some gain and of 
those 15 or 71% achieved mastery level. At Natter Mfg. . of 7 participants who attended at 
least 25% of the course. 5 or 71% achieved the mastery level. At National Semiconductor 
CorjL, 14 participants attended at least 25% of the course; of that 14. 13 or 93% showed some 
gain and 10 or 71% achieved mastery level. 

Workplace Communications and Team Building had a total of 41 participants who attended 25% 
of the course. Of that number. 28 or 68% showed gain and seven (7) showed mastery. While 
participants in Team Building showed gain in their awareness and knowledge of teamwork 
concepts, these are the types of skills that will show up later as employees have the opportunity 
to use the skills. 

Of the targeted 110 unduplicated participants for this objective, the majority (73 or 66%) 
showed gain. Other aggregate data that is available suggests that approximately 18 other ESL 
participants increased their communications skills. This increases the number to 91 or 81% 
of the target population. 
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Product Objective 6: increase participation of targeted employees in employee team 
meetings by 50%. 

Activities conducted to achieve the objective: 

The following classes were conducted: 
A. Natter Manufacturing, Inc. 



SESSION 
ESL 

Session 1 

Session 2 
Session 3 



CLASS 



DATES 



Fall ESL I 
Fall ESL II 
ESL BASIC 

SPRING ESL I 
SPRING ESL 11 

READ/WRITE 
CONVERSATION 



Oct. 1990 - Jan. 1991 
Oct. 1990 ' Jan. 1991 
Nov. 1990 - May 1991 

Feb. 1991 - May 1991 
Feb. 1991 - May 1991 



July 1991 
July 1991 



Sept. 
Sept. 



1991 
1991 



# SERVED 

(25%+ attendance) 



9 
8 
3 

7 
4 

5 
5 



COMMUNICATIONS 

Session 2 WORKPLACE Mar. 1991 - June 1991 12 

Session 3 WORKPLACE July 1991 - Oct. 1991 7 



B. National Semiconductor 
ESL 

Session 1 ESL 1 Sept. 1990 - Jan. 1991 11 

Session 2 ESL 2 Feb. 1991 - June 1991 9 

Session 3 ESL3 July 1991 - Jan. 1992 1 1 

TEAM BUILDING 

Session 3 TEAM BUILDING Nov. 1991-Jan. 1992 22 
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meetings by 50%. 



Explanation of discrepancy if necessary: 

There are some minor differences in the projected data collection in support of this objective 
and the actual data collected. The initial plan to observe at team meetings and establish a 
baseline response pattern which could be compared to post-training data and observation of 
team meetings throughout the period was not feasible because neither company had teams which 
met on a regular basis. Therefore, data to support this objective was drawn from hard data 
sources and soft data sources as noted. Self-evaluations and team evaluations were used in the 
Team Building class. 

Results: 

A. Natter Manufacturing, tnc. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

ESL-Enqlish as a Se cond Language -Teacher observation and anecdotal data indicated 
that seventeen (17) of eighteen (18) students enrolled in three classes in the Fall attended 
25% of the course. All seventeen (17) improved their skills in varying degrees. In the two 
Spring classes, eleven (11) of thirteen (13) students completed 25% of the course; of these 
1 1 . eight (8) showed some gain and seven (7) of those achieved mastery. The ESL Conversation 
class had five (5) students of the seven (7) enrolled who completed 25% of the course; three 
(3) showed some gain according to pre/post-test results and two (2) achieved mastery. The 
ESL Reading/Writing class had five (5) students who completed 25% of the course. Hard data 
was unavailable for this class. 

In terms of unduplicated participants. 21 of the 23 participants attended at least 25% of the 
course; 15 of those 21. or 71%. either showed some overall gain from pre- to post-test or 
improvement was noted and 7. or 33%. achieved mastery level on one or more post-tests. 

No data was collected in support of this objective. 



Projected Data Coiiection 

Observation at 2 team meetings 
to establish base (average 
of responses/meeting) 

Observation at meetings through 
out project period 



Actual Data Collected 

Individual Employee Files 

Assignments 

Pre/Post Test Results 

Supervisor Evaluations/ Surveys 

Training Program Feedback Surveys 

Self-evaluations 

Team Evaluations 
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Product Objective 6: increase participation of targeted employees in ennployee team 
meetings by 50%. 



Workplace Communications -Out of 20 enrolled, 19 (95%) attended at least 25% of 
the course. Of the 19, 14 or 74% of the participants showed some gain and 7 or 37% of those 
achieved the mastery level. 

No data was collected in support of this objective. 

B. National Semiconductor Corp. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

English as a Second Language -A limited amount of hard data in the form of pre-post test 
results is available for the ESL classes. Out of eleven (11) students enrolled in the ESL 1 class, 
11 (100%) attended 25% of the course. Teacher observation, anecdotal data and evaluation 
indicated that all 11 improved their skills in varying degrees. Out of 9 students enrolled in the 
ESL 2 class, 9 (100%) attended 25% of the course. The 9 students showed an average gain of 
13% on the post-tests for the objectives tested. Out of eleven (11) students enrolled in the ESL . 
3 class, 1 1 (100%) attended 25% of the course. In the ESL 3 class, 5 students who took the 
post-tests showed an average gain of 18% for the two units for which pre/post-tests were 
available. Data that is available suggests that gains were made by participants. 

No data was collected in support of this objective. 

Team Building -Out of 23 enrolled, 22 attended at least 25% of the course. Each 
participant did a pre and post self-evaluation and a team evaluation. Supervisors evaluated each 
team also. Results of self-evaluations and team evaluations completed by 14 , or 64%, of the 
participants who completed the course and three supervisors yielded results which indicated 
increases in awareness or knowledge of ten teamwork concepts. The data is presented betow. 

Pre-Post Self Evaluation: 

A self-evaluation form consisting of nine teamwork concepts which were to be ranked 
from 1 (low) to 5 (high) was administered to participants. The total average score was 3.52 
for the concepts. A post-self evaluation which used the same items asked participants to 
measure the percent of increase in their awareness and/or knowledge of those same concepts. 
The average percent of increase was 70.78 percent. 

Pre-Post Team Evaluation: 

A team-evaluation form consisting of 9 teamwork concepts which were to be ranked 
(5 =high) was administered to each participant to evaluate the team. The average score was 
3.24 for the concepts. A post-self evaluation which used the same items asked participants to 
measure the percent of increase in their awareness and/or knowledge of those same concepts. 
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Individual member evaluations of the team showed average increases of 62% to 77% for the ten 
teamwork concepts. The average percent of increase was 71.33 percent. 

Pre-Post Manager Team Evaluation: 

A team-evaluation form consisting of nine teamwork concepts which were to be ranked 
(5=high) was administered to each manager. The total average score was 3.00 for the concepts. 
A post-self evaluation which used the same items asked participants to measure the percent of 
increase in their awareness and/or knowledge of those same concepts. Supervisors' evaluations 
of the three teams' increase in awareness or knowledge of ten teamwork concepts showed average 
increases of 46%, 67% and 71% for an average increase of 61%. The total average percent of 
increase was 61.33 percent for the three teams. 

Survey Data: 

Training Program Feedback: 

This evaluatton was administered to participants at the end of the course. Items focused 
on instructor content knowledge and teaching skills, materials, program design, relationship of 
course content to job. etc. The items were ranked from 1 (low) to 5 (high). The total average 
score was 4.46. 

No data was collected in support of this objective. 
Discussion of the Results: 

Hard Data : Even though the structure and use of teams within the companies precluded 
the collection of hard data in the manner originally planned, results indicate that the training 
resulted in gains in communications skills. Gains were difficult to measure for ESL 
participants and relied heavily on anecdotal data and observation/evaluation by the teacher and 
supervisors. Team Building was conducted as a pilot course and due to the nature of the course, 
a pre-test of content knowledge did not seem an appropriate measure. The measure of success 
for such a course often comes far after the course ends and teams have more opportunity to put 
into practice the concepts they learned. Therefore, the measures used were highly subjective, 
i.e.. self-evaluations and team evaluations by participants and managers. These indicated that 
this was not only a very popular course, but the employees rated it very highly in terms of 
relationship/value to their jobs and the new direction the company is pursuing, i.e. , self- 
managed teams, and increases in their ability to express themselves in their teams, handle 
conflict, etc. This can be interpreted to mean that these behaviors will carry over into team 
meetings as the companies, especially National Semiconductor, continue with the goal of 
implementing self-managing teams. Hard data does not support the increase in participation by 
50% for targeted employees primarily because there were no consistent team meetings to 
measure participation. However, it can be surmised that employee participation did increase in 
other forms of oral communications on the job. 
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Product Objective 6: increase participation of targeted employees in employee team 
meetings by 50%. 



Survey Data: Data from the Training Feedback Survey indicates that employees valued 
the courses and believed the courses were well-taught, had definite relationship to their jobs, 
and provided them with useful skills. 

Soft Data : Results from survey data were overwhelmingly positive in terms of value 
and relationship of the courses to the employees' jobs. Workplace Communications and Team 
Building participants believed that all employees should take the course and stated that it helped 
them understand not only their co-workers better but also management's perspective. A 
significant number indicated the course enabled them to perform in ways they were unable to 
before the course. For example, responses indicated participants understood more about 
teamwork, could express themselves better in problem-solving activities, work better as a 
team member, understand why teams have problems and not take it personally, and 
communicate better as a team player. ESL participants reported increased understanding of 
their supervisors, increased speaking skills, and increased understanding of workplace 
language. Supervisors also reported the same kinds of skills increases and in some cases 
reported increased ability to handle more complex responsibilities, reduced time spent in 
explaining tasks to employees, less need for use of an interpreter, and increased productivity. 
Self- confidence and self-esteem were greatly enhanced by program participation as reported 
by participants and supervisors. 

Overall achievement: 

The objective was not met in terms of hard data that supports an increase of 50% in 
participation in team meetings. There was no way to collect the supporting data because the 
companies did not have teams which met on a regular basis. It can only be stated that there is 
evidence to support the claim that participants did increase their oral communications skills. 

From a total of 90 unduplicated participants, 55 (61%) showed gains in oral communications 
sKills or knowledge and awareness of skills necessary for successful team participation. Other 
anecdotal and aggregate data available suggests that approximately eighteen other ESL 
participants increased their oral communications skills. This increases the number of 
participants who showed gain to 73 or 79% of the total population. 
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Product Objective 7: increase employees' document literacy related to manuals, work 
orders, catalogues, etc. used on the work site to the 80% proficiency level based on pre-test 
results. 

Activities conducted to achieve the objective: 

The following classes were conducted: 
A. Natter Manufacturing, Inc. 



B. 



SESSION 
ESL 

Session 1 

Session 2 
Session 3 

MATH 

Session 1 

Session 2 



CLASS 



DATES 



Fall ESL I 
Fall ESL II 
ESL BASIC 

SPRING ESL I 
SPRING ESL II 

READ/WRITE 
CONVERSATION 



REFRESHER 
REFRESHER 

REFRESHER 
ADVANCE) 



READING & WRITING 

Session 3 TECH READ/WRITE 
National Semiconductor 
PRE-SPC 



Oct. 1990 - Jan. 1991 
Oct. 1990 - Jan. 1991 
Nov. 1990 - May 1991 

Feb. 1991 - May 1991 
Feb. 1991 - May 1991 

July 1991 - Sept. 1991 
July 1991 - Sept. 1991 



Oct. 1990 - Jan. 1991 
Oct. 1990 - Jan. 1991 

Feb. 1991 - May 1991 
Feb. 1991 - May 1991 



July 1991 - Oct. 1991 



Session 1 
ESL 

Session 1 
Session 2 
Session 3 



PRE-SPC 



ESL 1 
ESL 2 
ESL 3 



Sept. 1990 - Dec. 1990 



Sept. 1990 - Jan. 1991 
Feb. 1991 - June 1991 
July 1991 - Jan. 1992 



# SERVED 

(25%+ attendance) 



9 
8 
3 

7 
4 

5 
5 



12 
8 

5 
10 



141 



1 1 
9 
1 1 
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Product Objective 7: increase employees' document literacy related to manuals, work 
orders, catalogues, etc. used on the work site to the 80% proficiency level based on pre-test 
results. 



REFRESHER MATH 



Session 1 REFRESHER IMA Feb. 1991 - May 1991 11 

REFRESHER 2MA Feb. 1991 - May 1991 9 

REFRESHER3MA Feb. 1991 - May 1991 15 

REFRESHER4MA Feb. 1991 - May 1991 17 

REFRESHER5MA Feb. 1991 - May 1991 16 

REFRESHER 6MA Feb. 1991 - May 1991 15 



Sessions REFRESHER 1MB May 1991 - August 1991 15 

REFRESHER 2MB May 1991 - August 1991 6 

REFRESHER3MB May 1991 - August 1991 8 

REFRESHER 4MB July 1991 -Dec. 1991 8 

REFRESHERSMB June ^)9^ - Sept. 1991 9 

REFRESHER 6MB May 1 J91 - August 1991 8 



READING & WRITING 

Sessions TECH READAVRITE July 1991 - Nov. 1991 14 



Projected Data Collection 

Individual Employee Files 
Assignments 
Pre/Post Test Results 
Supervisor Feedback 

re use of materials 
Observation of skill 

transfer to job 



Actual Data Collected 

Individual Employee Files 
Assignments 
Pre/Post Test Results 
Supervisor Feedback 
re use of materials 
Observation of skill 
transfer to job 



Explanation of discrepancy If necessary: 



Supervisor feedback and observation of transfer to job were difficult to obtain for the same 
reasons noted for previous objectives. 



Results: 



A. Natter Manufacturing, Inc. 

Individual Employee Files/Assignments: 
On file. 
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Product Objective 7: increase employees' document literacy related to manuals, work 
orders, catalogues, etc. used on the work site to the 80% proficiency level based on pre-test 
results. 



Pre/Post Tests: 

Refresher Math- Out of 43 enrolled. 35 (81%) attended at least 25% of the course. 
Of the 35, 21 or 60% of the participants showed some gain and of those 1 1 or 31% of those 
showed gain to the mastery level . 

English as a Second Language -Teacher observation and evaluation indicated that 
seventeen (17) students enrolled in three classes in the Fall improved their skills in varying 
degrees. In the two Spring classes, eleven(ll) students completed 25% of the course; of these 
1 1 , eight (8) showed some gain and seven (7) achieved mastery. The ESL Conversation class 
had five (5) students who completed 25% of the course; three (3) showed some gain according 
to pre/post-test results and two (2) achieved mastery. The ESL Reading/Writing class had five 
(5) students who completed 25% of the course. Hard data was unavailable for this class. 

In terms of unduplicated participants, 21 of the 23 participants attended at least 25% of the 
course; 16 of those 21, or 76%, either showed some overall gain from pre- to post-test or 
improvement was noted and 7, or 33%, achieved mastery level on one or more post-tests. 

In relation to this specific objective for which data is available, seven (33%)participants 
showed gain and six (29%) of those participants achieved mastery. 

Technical Readinq/Writinq -Out of 7 enrolled, 7 (100%) attended at least 25% of the 
course. Of the 7, 5 or 71% of participants showed gain to the mastery level. 

In relation to this specific objective for which data is available, four (57%) participants 
showed gain to the mastery level. 

B. National Semiconductor Corp. 

Pre/Post Tests: 

Refresher Math - Out of 156 enrolled, 137 (88%) attended at least 25% of the 
course. Of the 137, 1 13 or 82% of the participants showed some gain and 85 or 63% of those 
showed gain to the mastery level. 

Pre-SPC -Out of 141 enrolled, 141 attended at least 25% of the course, Of the 141, 
91 or 65% of the participants showed some gain and 56 or 40% of those showed gain to the 
mastery level. Actually, only 106 participants took both pre- and post-tests. Based on this 
figure, 91 or 86% showed gain and 56 or 53% achieved mastery level. According 'o company 
records, 98% of the employees who took Pre-SPC achieved SPG Level 1 certification. 

English as a Second Language -A limited amount of hard data in the form of pre-post test 
results is available for the ESL classes. Teacher observation and evaluation indicated that 
eleven (11) students enrolled in the ESL 1 class improved their skills in varying degrees. In 
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Product Objective 7: increase employees' document literacy related to manuals, work 
orders, catalogues, etc. used on the work site to the 80% proficiency level based on pre-test 
results. 



the ESL 2 class, 9 students showed an average gain of 13% on the post-tests for the objectives 
tested. In the ESL 3 class, 5 students showed an average gain of 18% for the two units for which 
pre/post-tests were available. See discussion of results. The data that is available indicates 
that of 32 students enrolled, 24 of 26 participants who completed showed some gain. 

In terms of unduplicated participants, 28 of the 28 participants attended at least 25% of the 
course; 11 of those 28, or 39%, either showed some overall gain from pre- to post-test or 
improvement was noted. 

Although testing was conducted that related to this objective, individual pre/post-test scores 
were not available to document achievement of this particular objective. However, twelve 
(12) participants achieved an average increase of 16% in their scores from pre- to post-test. 

Technical ReadinqA/Vritinq -Out of 15 enrolled. 14 attended at least 25% of the course. 
Of the 14. 13 or 93% of the participants showed some gain and of those 10 or 71% of those 
showed gain to the mastery level. 

In relation to this specific objective for which data is available, eleven (79%) participants 
showed gain and eight (57%) achieved mastery level. 

Discussion of the Results: 

Hard Data: Results indicate that the training resulted in gains in document literacy. Increased 
ability to perform math functions required on the job in itself increases the ability to 
understand the documents used. Although not specifically tested, use of charts and graphs, time 
cards, etc. was included in the math and Pre-SPC curricula. Not ail Pre-SPC participants took 
both pre- and post-tests due to difficulties encountered during the start-up phase of the 
project; therefore, the results are skewed to the negative side when all participants who 
completed 25% of the course are included in the count. Gains were difficult to measure for ESL 
participants, but where pre/post test results are available they indicate improved ability to 
understand and use the documents related to their jobs. Anecdotal and other aggregate data 
suggests that more students made gains and probably achieved mastery than were included in the 
data available. Supervisors support this conclusion through anecdotal data. Technical 
Reading/Vyriting scores indicated gains from pre- to post-test in reading specifications used on 
the job and other kinds of charts and graphs. 

Soft Data : Results from survey data were positive in terms of value and relationship of 
the courses to the employees' jobs. Even in classes in which document literacy gains were not 
the primary goal and pre/post-test data was not collected, i.e.. Math and Pre-SPC, anecdotal 
data and self-evaluation/reporting by participants indicated that increases in this skill did 
occur. Math participants reported immediate use of skills learned/refreshed and equated it with 
increases in understanding their jobs better and enabling them to assume more responsibility, 
often on their own initiative, and increased accuracy in paperwork and math. They also equated 
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it with being able to see and solve problems on their own without assistance. Participants also 
reported being able to understand and fill out forms used on their jobs more accurately and 
seeing the value of accuracy. Pre-SPC participants indicated a better understanding of what 
SPC was ail about and that it made more sense to them as a result of the course. They also could 
see the value of it to the company and the need for accuracy in charting, math. etc. ESL 
participants reported increased understanding of workplace language. Supervisors also 
reported the same kinds of skills increases. Technical Reading/Writing participants reported 
increases in ability to write better, more concise memos, passdowns, and ECN reports and to 
read and understand specifications and work vocabulary. Self-confidence and self-esteem were 
greatly enhanced by program participation as reported by participants and supervisors. 
Supervisors noted participants showed increased job knowledge, applied new knowledge to the 
job, increased ability to handle the forms and paperwork used on the job, etc. Managers 
reported seeing an increase in the number and quality of memos, passdowns, etc. written by 
participants. 

Overall achievement: 

Overall, the objective was partially met. At National Semiconductor of 268 unduplicated * 
participants who attended at least 25% of at least one course, 201 or 75% showed some gain 
or improvement was noted and 139 or 52% achieved mastery level. At Natter Mfg .. of 47 
unduplicated participants who attended at least 25% of at least one course, 34 or 72% showed 
some gain or improvement was noted and 16 or 34% achieved mastery level. If the results 
included the eighteen other ESL participants, it is anticipated that the gain/mastery figures 
would be higher. 

Combined figures for both companies resulted in the following data: There were 315 
unduplicated participants in these classes. Of these 315, 235 or 75% showed some gain or 
improvement was noted and 156 (50%) achieved mastery. 
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Product Objective 8: increase writing skills used on the job for applications, tax forms, 
insurance forms, work orders, etc. 

Activities conducted to acliieve the objective: 

The following classes were conducted: 
A. Natter Manufacturing, Inc. 



SESSION CLASS DATES # SERVED 

(25%+ attendance) 

ESL 

Session 1 Fall ESL I Oct. 1990 - Jan. 1991 9 

Fall ESL II Oct. 1990 - Jan. 1991 8 

ESL BASIC Nov. 1990 - May 1991 3 

Session 2 SPRING ESLI Feb. 1991 - May 1991 7 

SPRING ESL 11 Feb. 1991 - May 1991 4 

Sessions READ/WRITE July 1991 - Sept. 1991 5 

CONVERSATTON July 1991 - Sept. 1991 5 

MATH 

Session 1 REFRESHER Oct. 1990 - Jan. 1991 12 

REFRESHER Oct. 1990 - Jan. 1991 8 

Session 2 REFRESHER Feb. 1991 - May 1991 5 

ADVANCED Feb. 1991 - May 1991 10 

READING & WRITING 

Sessk)n3 TECH READ/WRITE July 1991 - Oct. 1991 7 
B. National Semiconductor 
PRE-SPC 

Session 1 PRE-SPC Sept. 1990 - Dec. 1990 141 
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insurance forms, work orders, etc. 



skills used on the job for applications, tax forms, 



ESL 

Session 1 ESL 1 

Session 2 ESL 2 

Session 3 ESL 3 



Sept. 1990 - Jan. 
Feb. 1991 - June 
July 1991 - Jan. 



1991 11 

1991 9 

1992 1 1 



REFRESHER MATH 



Sessioni REFRESHER IMA Feb. 1991 - May 1991 11 

REFRESHER 2MA Feb. 1991 - May 1991 9 

REFRESHER3MA Feb. 1991 - May 1991 15 

REFRESHER 4MA Feb. 1991 - May 1991 17 

REFRESHER5MA Feb. 1991 - May 1991 16 

REFRESHER 6MA Feb. 1991 - May 1991 15 



Sessions REFRESHER 1MB May 1991 - August 1991 15 

REFRESHER 2MB May 1991 - August 1991 6 

REFRESHER 3MB May 1991 - August 1991 8 

REFRESHER 4MB July 1991 -Dec. 1991 8 

REFRESHER 5MB June 1991 - Sept. 1991 9 

REFRESHER 6MB May 1991 - August 1991 8 



READING & WRITING 

Sessions TECH READ/WRITE July 1991 - Nov. 1991 14 



Projected Data Collection 

Simulated writing Tasks 
Actual Writing Activities 
Supervisor Feedback 
re skill transfer 



Actual Data Collected 

Individual Employee Files 
Assignments 
Pre/Post Test Results 
Survey Data 



Explanation of discrepancy if necessary: 

Supervisor feedback and observation of transfer to job were difficult to obtain for the same 
reasons noted for previous objectives. 
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Results: 

A. Natter Manufacturing, Inc. 

Individual Employee Files/Assignments: 

On file. 
Pre/Post Tests: 

Refregher Math- Out of 43 enrolled. 35 (81%) attended at least 25% of the course. 
Of the 35. 21 or 60% of the participants showed some gain and 11 or 31% of those achieved the 
mastery level. 

English as a Second Language-Teacher observation and evaluation indicated that 
seventeen (17) students enrolled in three classes in the Fall improved ttieir skills in varying 
degrees. In the two Spring classes, eleven (11) students completed 25% of the course; of these 
11, d eight (8) showed some gain and seven (7) achieved mastery. The ESL Conversation class 
had five (5) students who completed 25% of the course and three (3) showed some gain and two 
(2) achieved mastery according to pre/post-test results. The ESL Reading/Writing class had 
five (5) students who completed 25% of the course. Hard data was unavailable for this class. 

In terms of unduplicated participants, 21 of the 23 participants attended at least 25% of the 
course; 16 of those 21, or 76%. either showed some overall gain from pre- to post-test or 
improvement was noted and 7. or 33%, achieved mastery level on one or more post-tests. 

Technical ReadingA^Vriiirig-Out of 7 enrolled. 7 (100%) attended at least 25% of the 
course. Of the 7. 5 or 71% of participants showed gain to the mastery level. 

B. National Semiconductor, Corp. 

Pre/Post Tests: 

Refresher Math- Out of 156 enrolled. 137 (88%) attended at least 25% of the 
course. Of the 137, 1 13 or 82% of the participants showed some gain and 86 or 63% of those 
achieved mastery level. 

Pre-SPC-Out of 141 enrolled. 141 attended at least 25% of the course. Of the 141, 
91 or 65% of the participants showed some gain and 56 or 40% of those showed gain to the 
mastery level. Actually, only 106 participants took both pre- and post-tests. Based on this 
figure, 91 or 86% showed gain and 56 or 53% achieved mastery level. According to company 
records, 98% of the employees who took Pre-SPC, achieved SPC certification. 

English as a Second Language-A limited amount of hard data in the form of pre-post test 
results is available for the ESL classes. Teacher observation and evaluation indicated that 
eleven (11) students enrolled in the ESL 1 class improved their skills in varying degrees. In 
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the ESL 2 class, 9 students showed an average gain of 13% on the post-tests for the objectives 
tested. In the ESL 3 class, 5 students showed an average gain of 18% for the two units for which 
pre/post-tests were available. See discussion of results. The data that is available indicates 
that of 32 students enrolled, 24 of 26 participants who completed showed some gain. 

In terms of unduplicated participants, 100% of the 28 participants attended at least 25% of the 
course; 1 1 of those 28, or 39%, either showed some overall gain from pre- to post-test or 
improvement was noted. Data for other individuals was incomplete or unavailable. Aggregate 
data suggests gain in skills for approximately 18 others. 

Technical ReadingAA/riting -Out of 15 enrolled, 14 attended at least 25% of the course. 
Of the 14, 13 or 93% of the participants showed some gain and 10 or 71% of those achieved 
mastery level. 

Discussion of the Results: 

Hard Data: Results indicate that the training resulted in gains in writing skills used on 
the job, even in classes in which this was not a primary goal, i.e., math and Pre-SPC. Gains 
were very difficult to measure for ESL participants and relied heavily on anecdotal data and 
teacher and supervisor observation of the transfer of skills to the job. Technical 
Reading/Writing scores indicated gains from pre- to post-test. Because writing was an integral 
part of the classes, gains measured in relation to other objectives can be applied to writing also. 

Soft Data : Results from survey data were overwhelmingly positive in terms of value and 
relationship of the courses to the employees' jobs. Math participants reported immediate use of 
skills learned/refreshed and equated it with increases in understanding their jobs better and 
enabling them to assume more responsibility, often on their own initiative, and increased 
accuracy in paperwork and math. They also equated it with being able to see and solve problems 
on their own without assistance. Participants also reported being able to understand better and 
fill out forms used on their jobs more accurately and seeing the value of accuracy. Pre-SPC 
participants indicated a better understanding of what "PC was all about and that it made more 
sense to them as a result of the course, they could see the value of it to the company and the need 
for accuracy in charting, math, etc. They reported increased ability to understand and use the 
charts. ESL participants reported increased understanding of workplace language. Supervisors 
also reported the same kinds of skills increases. Technical ReadingAA/riting participants 
reported increases in their ability to write better, more concise memos, passdowns, and ECN 
reports and to read and understand specifications and work vocabulary. Self-confidence and sell 
esteem were greatly enhanced by program participation as reported by participants and 
supervisors. Supervisors noted participants showed increased job knowledge, applied new 
knowledge to the job, increased ability to handle the forms and paperwork used on the job, etc. 

Overall achievement: 

Overall, the objective was partially met. At National Semiconductor Corp. of 268 
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unduplicated participants who attended at least 25% of at least one course, 202 or 75% 
showed some gain or improvement was noted and 139 or 52% achieved mastery level. At 
Natter Mfg.. of 46 unduplicated participants who attended at least 25% of at least one course. 
33 or 72% showed some gain or improvement was noted and 15 or 33% achieved mastery level. 
If the results included the eighteen other ESL participants, it is anticipated that the 
gain/mastery figures would be higher. 

Combined figures for both conpanies resulted in the following data: There were 314 
unduplicated participants who attended at least 25% of at least one course. Of these 314. 235 
or 75% showed some gain or improvement was noted and 154 (49%) achieved mastery. 
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Product Objective 9: increase quantitative literacy skills related to numerical operations 
performed on the job to the 80% proficiency level based on pre-test results. 



Activities conducted to achieve the objective: 

The following classes were conducted: 
A. Natter Manufacturing, Inc. 



SESSION CLASS DATES # SERVED 

(25%+ attendance) 

MATH 

Session 1 REFRESHER Oct. 1990 - Jan. 1991 12 

REFRESHER Oct 1990 - Jan. 1991 8 

Session 2 REFRESHER Feb. 1991 - May 1991 5 

ADVANCED Feb. 1991 - May 1991 10 

B. National Semiconductor 
PRE-SPC 

Session 1 PRE-SPC Sept. 1990 - Dec. 1990 141 
REFRESHER MATH 

Session 1 REFRESHER 1MA Feb. 1991 - May 1991 11 

REFRESHER 2MA Feb. 1991 - May 1991 9 

REFRESHER3MA Feb. 1991 - May 1991 15 

REFRESHER4MA Feb. 1991 - May 1991 17 

REFRESHER5MA Feb. 1991 - May 1991 16 

REFRESHER6MA Feb. 1991 - May 1991 15 

Sessions REFRESHER 1MB May 1991 - August 1991 15 

REFRESHER 2MB May 1991 - August 1991 6 

REFRESHER 3MB May 1991 - August 1991 8 

REFRESHER 4MB July 1991 -Dec. 1991 8 

REFRESHER 5MB June 1991 - Sept. 1991 9 

REFRESHER 6MB May 1991 - August 1991 8 
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Product Objective 9: increase quantitative literacy skills related to numerical operations 
performed on the job to the 80% proficiency level based on pre-test results. 



Projected Data Collection 

Pre/Post Test Results 
Simulated Tasks 
Application to job/use of 
equipment 



Actual Data Collected 

Pre/Post Test Results 
Assignments 

Individual Employee Files 

Supervisor Surveys 

Training Program Feedback Surveys 



Explanation of discrepancy if necessary: 

There are some minor differences in the projected data collection in support of this objective 
and the actual data collected. Simulated tasks are incorporated in the assignments given in class 
and in some instances were used as practice assignments only. Instructors solicited feedback 
from supervisors regarding he transfer of skills to the job and/or use of equipment. Training 
Program feedback forms, administered to participants at the end of the class focused questions 
on relationship/value of training to the job. 

Results: 

Copies of the pre- and post-tests administered to participants and the results are in the 
participants' individual files. 

A. Natter Manufacturing, Inc. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

Refresher Math - Out of 43 enrolled, 35 (81%) attended at least 25% of the course. 
Of the 35, 21 or 60% of the participants shewed some gain and 1 1 or 31% of those showed gain 
to the mastery level. 

B. National Semiconductor Corp. 

Individual Employee Files/Assignments: 
On file. 

Pre/Post Tests: 

Refresher Math - Out of 156 enrolled, 137 (88%) attended at least 25% of the 
course. Of the 137, 113 or 82% r-: he participants showed some gain and 86 or 63% of those 
achieved mastery level. 

Pre>SPC -Out of 141 enrolled, 141 attended at least 25% of the course. Of the 141, * 
91 or 65% of the participants showed some gain and 56 or 40%% of those showed gain to the 
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Product Objective 9: increase quantitative literacy skills related to numerical operations 
performed on the job to the 80% proficiency level based on pre-test results. 



mastery level. Actually, only 106 participants took both pre- and post-tests. Based on this 
figure, 31 or 86% showed gain and 56 or 53% achieved mastery level. According to company 
records, 98% of the employees who took Pre-SPC, achieved SPC certification. 

Discussion of the Results: 

Hard Data : Results indicate that the training resulted in gains in quantitative literacy 
levels. Math results indicate the most significant gains. Even though some participants did not 
achieve the mastery level, examination of the range of scores indicates that some who scored at 
very low levels on the pre-test achieved significant gains. Pre-SPC results also indicate 
significant achievements. This course, the first in the program, was beset by a series of 
difficulties such as scheduling conflicts, communications problems between the training 
department/literacy program and company supervisors which surfaced in a variety of ways. 
These problems continued in the post-testing process so that only 106 participants took both 
the pre- and post-tests. If the results of the post-test are examined on the basis of this 
information, the data shows significantly different results. These results are presented above. 

S oft Data : Results from survey data were overwhelmingly positive in terms of value 
and relationship of the courses to the employees' jobs. Math participants reported immediate 
use of skills learned/refreshed and equated it with increasen in understanding their jobs better 
and enabling them to assume more responsibility, often on their own initiative. They also 
equated it with being able to see and solve problems on their own without assistance. Pre-SPC 
participants indicated a better understanding of what SPC was all about and that it made more 
sense to them as a result of the course, they could see the value of it to the company and the need 
for accuracy in charting, math, etc. Self-confidence and self-esteem were greatly enhanced by 
program participation as reported by participants and supervisors. 

Overall achievement: 

Overall, the objective to increase the quantitative literacy skills of targeted employees 
was met. Both companies had a total of 264 unduplicated participants who attended at least 25% 
of at least one course; 198 or 75% of the participants showed some gain and 139 or 53% of 
those achieved mastery level. 

At Natter Mfg. > of 26 unduplicated participants who completed at least 25% of the course in 
Refresher or Advanced Math, 14 or 54% showed some gain and or 8 or 31%% achieved 
mastery level. At National Semiconductor Corp .. of 238 unduplicated participants who 
completed at least 25% of the course in Refresher Math, Pre-SPC or both, 184 or 77% showed 
some gain and 131 or 55% of those achieved mastery level. 
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Administrative Issues 



The partnership achieved or partially achieved ail but one of the goals. Goals that were 
only partially achieved included Product Objectives 3-9. These are addressed in detail 
in the section on Product Objectives. 

Staff development for faculty needs to be strengthened, especially in the areas of 
developing instructional objectives and test design. Some of the problems we faced 
centered around curriculum design and this was addressed part-way through the grant 
by hiring a part-time curriculum writer. 

USDE needs to allow longer start-up periods. The education process for the business 
partner is lengthy, yet it is critical to the success of the project. Laying the groundwork 
and establishing the buy-in by all levels at the business takes time and a lot of effort. 
Likewise, the learning curve is very high fo- the education partner-staff development 
for a functional context program is critical and learning about the business partner is 
lengthy also. In addition, a three-year grant period is more realistic. Eighteen months 
is just not long enough to plan and implement a project of this scope. 

Contrary to adult learning theory which states that adults like individualized, self-paced 
instruction, we found that many workers like this approach but also want the teacher- 
lecture format. Perhaps this has something to do with pre-conceived notions of the role 
of teacher as "imparter of knowledge" rather than that of facilitator with the adult 
learner in charge of his/her own learning. When we discovered this, we adjusted the 
teaching style to accommodate some "lecture" for a short period of the session and then 
went on to more individualized, self-paced learning methodology. 

Assessment is a continuing challenge in all adult learning programs. Traditional, 
standardized tests were not used in our program, but we faced the issues of validity and 
reliability and had to be content with face validity. This is an arena that needs more 
exploration and opportunities to share information. 

Cost-benefit analysis sounds good in theory, but in actuality this is very difficult to do. 
There are too many variables within the business that affect productivity, scrap rate, 
etc. A very good program may not have any impact on these measures due to variables 
not within control of the education partner. Perhaps we should begin to look at other 
kinds of impact on the company such as changes in who receives the training and the 
training dollars (management or workers), changes in the way training is administered 
by the business because of spin-off learning from the education partner about how to 
conduct training, etc. 

CONCLUSIONS 

The project was successful. Both companies were pleased with the outcomes of the 
project and noticed considerable changes in many employees. The functional-context 
approach is better received than traditional ABE programs by both management and 
employees. Both education and business partners learned a great deal from each other 
and believed that it was truly a partnership. Future project leaders should be aware that 
the learning curve is very high on both sides of the partnership during the initial phases 
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of the grant. Project directors should take time to lay effective groundwork with all 
parties at all levels before launching into classes. Failure to do so will probably cause 
many problems and perhaps doom the project to failure. 

Staff development should not be underestimated. Many fine teachers wi!l have difficulty 
adjusting to the new challenges of the functional-context approach. Minimally, review 
of such theoretical bases as lesson planning, developing objectives, developing a course 
outline, designing activities, and developing criterion-referenced tests should form the 
base for staff development. More extensive staff development regarding this approach 
may be necessary depending on faculty needs. In addition, differences between workplace 
literacy and general literacy, "reading to learn" vs "reading to do," the teacher's role in 
workplace literacy, etc. should also be covered extensively in staff development. Another 
aspect of staff development should include training on the importance of tracking and 
documenting student progress. Historically, the field of adult education has not accepted 
responsibility for gathering hard data regarding student progress. It is time for the 
practitioners in the field to shoulder this responsibility and do more than say "I know 
he/she made progress" and yet have no data to back up that statement. 

An unexpected outcome of the project occurred in the area of family literacy. Many 
employees stated that they had increased their involvement in such activities as helping 
with homework, reading to their children, becoming involved in school activities, etc. 
Thus, participation in workplace literacy has a definite impact on the literacy 
development of chiidren, possibly increasing the value of literacy/learning in the home 
and breaking the cycle of intergenerational illiteracy. This is an aspect of workplace 
literacy programs that should be documented in other programs and used as a 
springboard to develop programs which will increase work-family literacy connections. 

RECOMMENDATIONS 

National Workplace Literacy programs should be continued. This partnership has been 
valuable to business and education partners alike. 

Extend the grant period to a minimum of three years. Eighteer: months is not enough time 
to start, develop, and complete a project. There is not time enough to really analyze what 
is occurring and in many instances see the results of the classes. Over a longer period of 
time supervisors would probably be able to see more long-term changes in behaviors 
and skills. Additional evaluation methodology might be developed if there were a longer 
time period. 

Appoint a task force/working group to explore assessment and testing issues in relation 
to workplace literacy. This group could collect samples of functional context, criterion- 
referenced assessmentS/'tests, portfolio assessment, etc. that have been used in projects 
and then develop a list of assessment/test resources, write a position paper on the status 
of assessment/testing in workplace literacy and future goals. 

Dissemination Activities 

As part of our project we conducted the following dissemination activities: staff of SLCC 
made presentations at six conferences, distributed project information at another 
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conference, presented to three in-house meetings or groups, and appeared on a local 
radio show. The project was cited in one publication, an article was published in a 
national policy study, an article was printed in a local newspaper, and two other articles 
were published in a booklet distributed at a conference. 

Presentations 

^ November 1990: Salt Lake Community College 

Chief Executive Officers' Round Table Breakfast 
"Workplace Literacy" 

Julie Van Moorhem. John Anjewierden, Blair Carruth 

' April 1991: Sun Valley, Idaho 

Mountain Plains Adult Education Association Conference(MPAEA) 
Workplace 2000: Panel Discussion 
"Building Industrial Leaders Through Literacy" 
Mary Mellott and Julia Balcom 

"Workplace Literacy: Gaining Supervisor Support" 
Julie Van Moorhem 

* June 1991: Logan, Utah 

June Vocational/Applied Technology Conferen^^:? 
"Workforce Literacy: What Are We Doing?" 
-Julie Van Moorhem 

" September 1991: Salt Lake City 
KSL Radio Show 

"Workplace Literacy" 

SLCC-Mary Mellott, Blair Carruth, John Anjewierden 
Natter Manufacturing, Inc.-Joy Spindler, Maurine Simmons 

" October 1991: Montreal, Canada 

American Association of Adult. Continuing and Community Education 
International Conference (AAACE) 

Distributed SLCC Workplace Literacy Project Descriptions 

Julie Van Moorhem 

* October 1991: Park City, Utah 

Utah Association for Adult, Community and Continuing Education Conference 
(UAACCE) 

"Workplace Literacy in the 90's" 
Julie Van Moorhem 

" November 1991: Albuquerque, New Mexico 

Rocky Mountain Region VIII Teachers of English as a Second Language Conference 
(TESOL) 

"Designing a Workplace Curriculum for ESL Employees" 
Mary Mellott 
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* December 1991: Salt Lake Community College 

Salt Lake Community College Board of Trustees 

"Workplace Literacy: A Partnership for Success" 
Julie V, Van Moorhem 

* March 1992: Kansas City, Missouri 

Midwest Regional Reading and Study Skills Conference 
"Workplace Literacy: Partnerships for Success" 
Julie Van Moorhem 

^ May 1992: Salt Lake City, Utah 

Mountain Plains Adult Education Association Conference (MPAEA) 
"Workplace Literacy: Partnerships for Success" 
Julie Van Moorhem 

Articles 

An article appeared in a local newspaper. The Salt Lake Tribune, November 1991. 

In addition, at least two curriculum books developed during the project will be sent to 
ERIC. They are currently being proofread/edited and permission of the companies 
obtained before they are submitted. 

The project is listed in Literacy at Work: The Workbook for Program Developers by 
Jorie Phillipi, published by Simon & Schuster Workplace Resources and the National 
Alliance of Business, as a source of information regarding planning and conducting the 
kinds of functional context activities described in the book. 

The project director, training coordinator at National Semiconductor, and the internal 
evaluator submitted an article to the Work in America Institute for inclusion in their 
national study entitled Job-Linked Literacy Programs: Innovative Strategies at Work . 
This was published in Spring 1992. 



Evaluation Activities 

This project was unique m that two evaluators, internal and external, were used. The 
internal evaluator had input at various points throughout the project and evaluated 
curriculum manuals, student files, and monitored time lines for the project. The 
external evaluator made three site visits and monitored curriculum development 
activities, achievement of goals, and conducted interviews with employee participants, 
supervisors, and management at each company. 

A variety of instruments were developed for use in evaluating the project. Teacher made 
assessment instruments, survey instruments, and evaluation instruments were designed. 
These were administered across ail employee levels during the project period. The 
instruments are listed beiow: 
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* Training Program Feedback Ai the close of each class, a Training Program 
Fedback form was administered to participants. This evaluation focused on four major 
areas of the program: General Feedback (description and purpose of training); the 
Trainer (knowledge, teacning ability,); Training Materials; Program Design; Other 
(like/dislike program, relationship of program to job and other training opportunities, 
suggestions for improvement). The items fit one of two forms-a rating scale (5=hig^l) 
or comments/suggestions. 

* Employee self-evaluations, supervisor evaluation of employee, and interviews 
with employees at entry-level, supervisor, and management levels were conducted also. 

* Post-Program Participant Survey was administered to participants at the close of 
the program. Demographic data and program data were gathered. The survey focused on 
questions in the following areas: Skills acquired through the program that were not 
present before participation; achieving personal goals (promotion, bid on new job, 
filling out accident/sick papers for work, being recognized/appreciated, understanding 
the job better); achieving educational goals (participate in/complete QED courses, 
enroll in college courses or professional training outside of work, increased 
participation in company training programs); increase community involvement (voting, 
reading newspaper, helping children with home work, reading, etc., volunteering in 
community/school programs); recommending program to co-workers; recommending 
program changes. 

* Post-Program Supervisor Rating was administered to supervisors at the close of 
the program. Supervisors ranked employee participants on a 4-point scale (4=high). 
Items focused on issues such as: changes in employee behaviors (quantity/ quality of 
work, job attitude, attendance, job knowledge); promotion/raise recommendation if 
economic conditions permitted; change in supervisor's confidence in the employee's 
performance; employees asking about opportunities for promotion, other positions, 
assuming additional responsibility, etc. 



EVALUATION MATRIX 



Self 



Employee Instructor Course 



Program 



Employee 



X 



X 



X 



X 



Supervisor 



X 



X 



X 



Trainer/Instructor 



X 



X 



X 



Evaluators 



X 



X 



X 
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NATIONAL WORKPLACE LITERACY PROGRAM 
INFORMATION FORM 



Part 1: Program Parameters 

225 

1. Target No. to be Served: 



2. No Served at Each Site (o Date: 

Site 1. 
Site 2. 69 



Site 3- . 
Site 4. . 
Site 5. 



Site 6. 

Site 7. 

Site 8. 

Site 9. 

Site 10. 



3. Total No. Served: 



35R 



4. Fed. Funds Obligated: $392,733.0 0 

5. Matching Funds/ ln-Kind:$ 289,960. 62 

Includes value release tirne 

6. Value Release Time: 

7. No. Participating in Programs 
Offered : 

Basic Skills ^05 

GED 

ESL 



8. Contact Hours Provided: 11/983 

(Contact Hours are the number of 
teaching hours that workers 
receive) 



Part 2: Participation Data 
1. Mean Age Participants: 



3. Race/ Ethnic itv: No. who are: 



White 
Black 



237 



Hispanic 39 



Am. Indian/ 

Alaska Native i 

Asian/Pacific 
Islander ^6 



6. Outcomes 



[Mo Participants 



a. Tested higher on basic skills 

b. Improved communication skills 

c. Increased productivity 

d. Improved attendance at work 

e. increased self-esteem 



27P 



73 



2. Sex: No. Males L^^No. Females 

4. No. Single Head of Household: ^8 



5. No. Limited English Proficient: 



53 



7. Years with the company Ng. Part>gipan!s 



Unemployed 

0-5 

6-10 

11-15 

1 6-over 



7A 



53 



33 



